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ABSTRACT 



Having a \Wtcle variety of outcome information without any structure is analogous to possessing 
a file cabinet in which the contents are arranged randomly. Simiiariy, without agreement on a 
common language and coniext for outcomes, it is difficult for institutional officials to 
communicate succinctly how their institution anjd programjj lifers from Its^counterparts.^An 
effective outcomes structure can be of assistance to postsecondary education planners and 
managers for those purposes as well as for identifying needs, developing goals, translating 
goals into concrete objectives, setting priorities and plans, and evaluating institutions and their 
programs. 

The potential volume of outcomes information in postsecondary education is quite formidable 
and poses a significant barrier to urging outcome information in institutional planning and 
management. Although a number of attempts have been made to develop structural systems for 
organizing outcomes information, all of them have so far proved inadequate to Meet the 
practical need. Because of the limitations in the state of the art and the important planning and 
management needs that exist in this area for postsecondary education, an NCHEMS project 
was conceived and executed that aimed to: (1) develop a set of coherent and wiclely accepted 
outcome constructs that describe what an "educational outcome" is, and (2). based on the 
conceptual framework provided by those constructs, develop a system that can be used to 
organize outcomes information In an effective way for purposes of classification, analysis, and 
decision making. 

The end product for the project was to be an "outcomes structure," a framework for organizing 
and classifying information about the full range of postsecondary education outcomes. Basic 
to the development of the Outcomes Structure, however, was the development of an 
appropriate conceptual framework. Until the present time, the term "educational outcome" has 
meant different things to different people. Therefore, based on an extensive synthesis and 
analysis of relevant literature, a set of six attributes or characteristics of an "educational 
outcome" was formulated: (1) Form, (2) Change Status, (3) Focus, (4) Neutrality, (5) Measur- 
ability, and (6) Output/Impact. Five outside factors are also important in understanding the 
concept of "educational outcome": ^1) Producer/Facilitator, (2) Audience, (3) Intended/ 
Unintended, (4) Functional Area, and (5) Time* The Outcomes Structure has three formal 
dimensions, where an outcomes dimension is a continuum that can be divided into segments 
along which outcomes can be placed and viewed in relation to one another. The three 
dimensions are: (1 ) Audience— the perspns, groups or entities that receive and/or are affected 
by (or which are intended to receive or be affected by) the outcome of concern; (2) Type of 
Outcome— whether or not the outcome involves a change in status (maintenance versus, 
change) and the basic, specific entity that is maintained or changed; (3) Time— the time frame 
in which the outcome occurs or is intended to occur. 

The Outcomes Structure has been tested out primarily through the use of logic. In addition, 
independent judges classified a sample of outcome statements to check for reliability in 
classification, the Structure was used in a preliminary way at one public university and eight 
small private colleges, and various practitioners in different types of postsecondary education 
settings have reacted to the Structure. Also, a review version of this document was tried out as 
a basic text in a graduate student course on outcomes at another university. Based on these 
experiences, it is fpit that the Structure has a number of potential uses throughout 
postsecondary education, but an indication of its real usefulness awaits practical tryout within 
postsecondary education institutions and at other levels. 
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FOREWORD 

This document describes the development of a systenn designed to organize information about 
intended and/or actual postsecondary education outcomes in an effective way for purposes of 
classification, analysis, and'decjslon making. It is intended for use by researchers working in 
the area of educational outcomes, by administrators, and by other educational practitioners 
who desire a detailed and in-depth discussion of the NCHEMS Outcomes Structure and of the 
concept of "educational outcomes.** Another document titled The Outcomes Structure: An 
Overview and Procedures for Applying it in Postsecondary Education Institutions has been pre- 
pared for day-to-day use by administrators and other educational practitioners. This Outcomes 
Structure has been developed as a part of the NCHEMS Outcome Structures and Measures pro- 
ject, supported by funds from the National Institute of Education. The earliest forerunner of the 
Outcomes Structure is the NCHEMS Inventory of Higher Education Outcome Variables and 
Measures (Micek and Wallhaus, 1973). More recently, a preliminary paper (Service, 1974) sum- 
.marized some initial conceptual work with respect to the Outcomes Structure. This paper pro- 
vided a basis for review and discussion by NCHEMS staff, the Outcomes of Postsecondaiy 
Education Project Task Force, and other interested parties. Results of these discussions were 
incorporated into the development plan that led to the publication of this document. 

As part of thai development process, two extensive literature review efforts were inaugurated 
late in 1974. Yong S. Lee explored the meaning of the concept "outcome,** and attempted to 
derive a definition for this concept that would be appropriate for planning, management, and 
policy-development purposes. Various people have viewed postsecondary education outcomes 
in different ways, and a special concern was whether one definition of "outcome** would suffice 
for the Structure or whether the definition would have to be adjusted according to the context in 
which the term was being used. 

In the second literature review Oscar T. Lenning explored: (1) literature in the field of taxonomy 
for principles or criteria that should be consideied in developing a classification structure for 
the outcomes of postsecondary education, (2) the literature describing previous attempts at 
classifying educational outcomes and outcome-related concepts such as goals and objectives 
(more than eighty such classifications were eventually found), and (3) the literature on specific 
postsecondary education outcomes that could be used to generate a broad list of outcomes for 
use in testing the Outcomes Structure resulting from the project. Both the Lee and Lenning 
reviews were continuations and extensions of the comprehensive review of the research and 
theoretical literature conducted by Lenning and associates at the American College Testing 
Program (1974, 1975). 

With these literature reviews as background, a draft version of the Outcomes Structure was 
developed, this draft was examined in detail by NCHEMS staff and selected external consul- 
tants. Subsequent revisions of the Structure also were tested by logical analysis, and the 
Structure was tried out in a practical way at one public university and eight small private col- 
leges (see pages 38-43). In addition, the Outcomes Structure was subjected to an extensive 
review by a Design Review Committee formed especially for that purpose. The committee in- 
cluded representatives of various sectors of postsecondary education as well as different types 
of potential users and appropriate methodologists. A variety of other postsecondary education 
practitioners also reacted to the review edition of this document that was completed in the fall 
of 1976. 

In conclusion, the NCHEMS Outcomes Structure is an evolutionary product. It rests upon 
significant bodies of previous work and is intended to continue to grow and develop as time 
passes. 
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Throughout this document there are examples of potential outcomes claimed by various people 
to be products of postsecondary education. NCHEMS does not warrant that these outcomes 
are products of postsecondary education. {There is strong research evidence, however, that 
some of them are.) For example, various student outcomes aimed/ for by Institutional and 
program practitioners may be primarily the result of experiences away from the campus— such 
as in ihe community surrouncJing the cciiege or at home with parents, siblings, and friends. 
Similarly, intended outcomes for young peoplo may result from the natural maturationaf 
processes of growing older, and might occur even if the person entered the world of work out of 
high school rather than entering postsecondary education. This is not saying, however, that 
postsecondary education experiences have no potential, for hastening the occurrence of such 
outconfies. Whether or not outcomes result from postsecondary education is not the concern of 
this document— research on that queotion is continuing at NCHEMS and elsewhere. Rather the 
concern of this document is with improving the ability of people to communicate more 
effectively about such outcomes, and to organize, analyze, and use outcomes information in 
decision making— whether the outcomes be actual, planned, or simply desired. " 
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Chapter 1 
OVERVI EW AND CONTEXT 

4 



Introduction 

All the major activities conducted by administra- 
tors, faculty, and student personnel workers in 
postsecondary education institutions ari^, pre- 
sumably, aimed toward bringing about certain 
kinds of "educational outcomes." The activities of 
planning, budgeting, .management, evaluation, 
research, scholarship. Instruction androTher 
developmental activities for students, and even 
staff development, all have educational outcomes 
as a primary focus. Other groups within an 
educational institution, suoh as the secretarial 
staff and the computer operators,''-' also 
presumably contribute to various educational 
^ outcomes. 

By "educational outcomes" we mean more than 
the effects of postsecondary education on 
students and the further impacts of those on 
others, including society in general. As used here, 
"educational outcomes" refer to any results or 
consequences of an cklUcational institution and 
its progtams. The outcomes ifnay be direct results 
of institutional activities, such as academic 
^d^grees, technological discoveries, student 
knowledge and skills, or institutional staff 
salaries. Conversely, there may be later 
consequences of those outcomes, such as 
individual prestige, higher family income, more 
educated work f 6rce,.or effects of staff salaries on 
the localeconomy* 

Educational activities focus on intended 
outcomes, but unintended or unplanned 
outcomes should also be of concern to educators. 
Unexpected or unintended outcomes can 
occasionally become more important than, the 
intended outcomes. Some unintended outcomes 
are considered to be of positive value by most 
people tf or example, increased student ingenuity 
or creativity), but othei; kinds of outcomes are 
considered to be detrimental (for exav^nple, 
increase<f drug use or political radicalism). 

Actually, the generic concept^ of "educational 
oqtcome" is a neutral one, separated from any 
inherept value status. But people attach value 
connotations to the outcomes, and even the most 



universally accepted educational outcomes are 
probably seen as negative by some people. 

As discussed in^later^chapter of this document,' 
it is important, in exploring outcomes, to separate 
the direct outcomes of institutional activities from 
the later consequences of those outcomes. 
Similarly, it is important to distinguish intended 
from unintended outcomes, and to be aware Oi to 
whom the outcome appeals and does not appeal* 
There are also other distinctions that can help 
institutional officials and others to understand a 
particular educational outcome: whether' it 
involves maintenance (preserving, replenishing, 
reproducing, or stabilizing) or change (modifying, 
revising, enriching, restructuring, or replacing); 
whether the outcome takes the form of a product, 
an event, or a condition; the measurability of the 
outcome; who or what receives or is affected by 
the outcome, and in what functional area; the 
specific institutional activities coj^tfjbuting to the^. , 
outcome; and when the outcome VCcurs and fk>w!t ^ 
long it lasts. ^. ' 

Because educational outcomes, are Uie raison 
d'etre of the educational endeavor^" it"is"{Hiportdnt 
to have a common language for comfpunicating to 
one another about outcomes, alnd to have 
information available about whidh outcomes are 
occurring and to what extent they are occurring. 
Equally important is the ability to interpret and 
use outcomes infomnation. 

This document is a description and discussion of 
a mechanism desired to aid in communicating, 
fnterpreting, and using information about 
postsecondary education outcomes. The mecha- 
nism consists of a proposed outcomes structure 
(a system for separating outcomes into meaning- 
ful categories) for postsecondary education and 
some general guidelifies for its use. The NCHEMS 
Outcomes Structure is intepded as ari alternative 
organizatidnal framework that will accommodate 
information about the full range of postsecondary 
education outcomes. 

The emphasis within this Classification scheme is 
^ upon organizing, differentiating, and showing 
relationships among the variety of educational 
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outcomes. Individual outcomes can be located 
within the classification structure and certain 
relationships with other outcomes assessed. The 
NCHEMS Structure was bMilt on an extensive 
foundation of previous work and is believed to be 
useful, but whether its use will yield general 
practical benefits awaits its application at 
postsecondary institutio ^s across the country. It 
Is expected that improvements and modifications 
of the Structure will be made in the future, based 
on continued deveiopmental activities and" on 
actual experience with using theStrycture. 



The Context and Need for an Outcomes Structure 

There is considerable evidence of continuing and 
increasing interest in understanding and 
. documenting the outcomes of postsecondary 
education Institutions. Individual inistitutions are 
faced with a variety of challenges.arid pressures 
that require them to go weir beyond, their 
^aditional'data-collection and disserhination 
, activities. The generation and use of meaningful 
outcomes inforniation is one key element of this 
task. The need at the institutional level is 
reinforced and augmented by an increasing 
competition for societal resources that provides 
an impetus for postsecondary education as' a 
whole to become more capable of expressing its 
contribution to individuals and society. 

The outcomes of postsecondary education can be 
viewed in a number of different contexts. In the 
largest sense, one can talk about the aggregate 
outcomes of the entire system of postsecondary 
- education, that is, the products postsecondary 
education provides the society, and nation (and 
the world) and the impacts of those products. 
Next one could talk about the outcomes of 
institutions of particular types, such as state 
university systems, liberal arts colleges, 
community colleges, vocational schools, and 
proprietary institutions. There are also different 
contexts at the individual institution level.. The 
college president focuses on institution-wide 
outcomes, the department head focuses on 
departmental outcomes, and the instructor 
focuses on the outcomes of the particular courses 
he or she is teaching. Outcomes are important at 
all of these levels and there is a commonality in 
the concept of "outcome." However, there are 

^ H should be noted that although the Outcomes Structure was designed 
specificafty for use at the postsecondary level, it n^y have utility also at the 
elenrtentary and secondary levols of education, ' 



also differences for different levels, for example, 
in the types of outcome information desired for 
decision making and in the ways that outcome 
information is gathered and interpreted. 

It is possible to view the collection and use of 
meaningful information about specific outcomes 
as involving at least five interrelated tasks: 
(1) identifying intended and potential outcomes 
and associated measures, (2) gathering outcomes 
data, (3) structuring the data, (4) analyzing the 
data, and (5) applying the results of the analysis 
toward modifying and improving the institution 
and its outcomes. 

For the first task, the range of postsecondary 
education outcomes to be considered should be 
identified as a prerequisite to their measurement. 
This involves transforming mission statements 
and needs assessment data into goals, which in 
turn are. translated into more specific, concrete 
otDjectives stated in outcome terms. ^ It also 
involves logical consideration of what significant' 
(for planning purposes) .unintended outcomes 
have the potential to occur, and their likelihood of 
occurring, then, to the greatest extent feasible, 
one or mgre measures that can serve as 
quantitative indicators should be associated with 
each potential outcome. 

For the second task, the measures identified, are 
applied through the definitions and procedures 
necessary for acquiring outcomes information, 
and the appropriate data are collected. The 
NCHEMS Outcome Measures and Procedures 
Manual (Micek, Service, ar^ Lee, . 1975) is^ 
intended as a support for exactly this kind of J 
measurement activity. o 

A third aspect of the outcomes information , 
question can be viewed as a structural' task. 
Organizing the array of educational outcomes in 
some conceptually sound and operationally 
useful manner is an important element in the 
productive use and understanding of outcomes 
data. At the very least, a structure can provide a 
basis for analysis of the interrelationships among 
various outcomes and also serve as a vehicle for 
more coherent communication of outcomes 
information among different users and decision 
makers. One version of such a structure is the 
NCHEMS Inventory of Higher Education Outcome 
Variables and Measures (in Micek and Wallhaus, 

' General outcome goals can be generated In an open-ended manner or with 
the aid of an Instrument like the ETS Institutional Goals Inventory. 
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1973). Many other attempts to structure 
educational outcomes also have been made 
(Lenning, 1977b). . - 

0nc3 the outcome data have been identified, 
generated, and structured, analysis appears as a 
logical fourth task. Such analysis might be 
directed toward identifying redundancies among 
outcome measures, assessing the^'validity of the 
data, identifying tentative relationships between 
outcome measures and other variables, and 
identifying and' understanding the outcomes that 
have occurred. ^ 

Once the dat^^^ have been analyzed and the 
outcomes ascertained, the findings need to be 
applied to the institutional decision-making 
process. Decisions about institutional and 
program modification depend on such informa- 
tion, as do general policy and management 
decisions. Students' and funders' decisions abou^ 
institutions and programs can also be facilitated 
by such information. Procedures and guidelines 
are needed, however, to integrate the outcomes 
information wifh information about inputs and 
processes and to apply such information to 
planning and management problems in an 
effective manner. 

This document is directed to the first and third of 
these five tasks, outcome identification and 
structuring. As mentioned previously, the 
structure discussed here was designed to help 
identify, organize, and classify information about 
the iull range of possible postsecondary 
education outcomes. 

The need for an outcomes structure arises from 
practical as well as conceptual considerations. 
Such a device can stimulate people.to think about 
outcomes in a more systematic and concrete 
manner than was previously the case. Also, the 
potential volume of outcomes information is quite 
formidable. The sheer mass of this body of data 
could well be a significant barrier to the overall 
objective of use of outcomes information in 
planning, and management. In the absence of 
some framework or structuring device, data item^' 
of interest must be identified individually; there is 
no mechanism for aggregating or for referencing 
a series or class of items. Having a wide variety of 
outcome information without any structure is 
analogous to possessing a file cabinet in which 
the contents are arranged randomly. The ability to 
retrieve and communicate the contents of the file 
improves as the organization of the material 



within it becomes more explicit. Therefore, an 
explicit organizing scheme is needed to aid iri 
organizing, filing, and retrieving outcome 
information. Such a framework could conceivably 
serve as the basis for the outcomes portions of 
computerized information storage and retrieval 
systems. 

A closely related need can be described in terms 
of opportunities for effective communication of 
outcomes information. Without some agreement 
t)n a common language or context, there is a very 
serious risk that particular outcome measures will 
assume different meanings from one situation to 
another. Given the imperfections of measurerftent 
technology, some ambigu|ties'wlil undoubtedly 
always remain.- Nevertheless, an outcomes 
structure should help to minimize or at least 
reduce this phenomenon by delineating a set of 
concepts and relationships that remain constant ^ 
from one use of.outcpmes information to another. 
In a very real sense, the outcorries structure can 
provide a grammar that links together individual 
words (outcomes information items) and thereby ~ 
creates a workable language for communication ^ 
about postsecondary education outcomes. This 
language would assist individuals In postsecond- 
ary education in sorpewhat the- same way that 
Linnaeus's taxonomy provided "a common 
language and helped biologists to« identify and 
categorize the various hierarchies of living; 
organisms. Once the taxonomy was completed, 
biologists were in a better position to itjentify, 
measure, and analyze the characteristics and 
changes among the various species. As a result; 
their understanding and comprehension of living 
organisnris increased, and their communication 
with other scientists improved. 

Another need is for some de.yice'to assist in the 
identification of those outcomes (or categories of ^ 
outcomes) for which adequate quantitative 
measures are not available. The process 'of 
associating measures with elements of an 
adequate outcomes structure could, presumably, 
fead to identifying areas in which quantitative 
information is deficient or altogether lackihg. 

There are additiona! needs that an outcomes 
structure could help^to meet. One is the need for 
postsecondary education institutions and their 
sub-units to improve their goal definitions and to 
translate more effectively their goals into concrete 
objectives. Another is the need for planners and 
managers to communicate more succinctly how 
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their institution or program differs- from its 
counterparts. Stiii another need is for a framewock 
that can guide the assessment .of the types 6f 
outcome information needed by different decision 
makers (trustees, presidents, academic pi'anning 
administrators, student affairs administrators, 
budget and finance administrators, iegisiators, 
directors of state agencies, and so forth), Finaiiy, 
there is a need for anaiyzing and evaiuating the 
outcomes of institutions and their programs. As a 
case In point, a comprehensive outcomes 
ciassification structure couid be of assi^ance to 
an evaiuator using a **goai-free evaiuation" 
approach (an evaiuation in which the actuai 
outcomes of an institution or program are 
identified without any prior knowiedge of the 



outcomes that were intended) by providing an 
exhaustive listing of aii the outcomes that might 
be awieved (positive or negative). 

Researchers in education also have important 
needs that an outcomes structure couid help to 
meet. Use of such a structure might help to 
determine the relationships among outcomes and 
classes of outcomes, to identify gaps in the 
comprehensive set of outcomes, and to enhance 
measurement capability with respect to particular 
outcomes or classes of outcomes. Research and 
development work in the outcomes area is far 
from complete, and a well-designed outcomes 
structure could lend viable support for future 
efforts. 
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Chapter 2 
A CONCEPTUAL FOUNDATION 



FOR THE 
OUTCOMES 



NCHEMS 
STRUCTURE 



Introduction 

A primary objective of the NCHEMS Outcome 
Structures and Measures project was development 
of a practical and useful classification system that 
encompasses the full range of outcomes. Since 
any such structure rnould come out of a 
conceptual and theoretical base, development 
required some resolution of the difficult question 
of exactly what should be included binder the 
rubric "outcomes of postsecondary education'* 
and what should be excluded. At the present time, 
the term "outcome" means different things to 
different people. For some the term denotes 
"output" (Goodman,. 1971) or "planned output" 
(Hoenack et al, 1974). For others the same term 
signifies "end results" or "ultimate consequences^' 
(Robinson and Majak, 1967; most PPBS 
literature). For still others the term stands for 
intended benefits (Hitch, 1970; Becker, 1964) or 
conversely^^or unintended effects or "side effects" 
(Bauer, 1966; Cook and Scroll, 1972). "Pro- 
ductivity"— maximizing outputs obtained from a 
given amount of resource inputs or minimizing 
inputs needed to produce a given amount of 
output—was the concept of "outcdme" empha- 
sized by Hitch (1970) and Christenson (1969), 
while Astin (1970) has focused not on maximizing 
outputs with respect to inputs but on comparing 
output conditions, characteristics, and levels to 
those at input (value added). Other frequently 
used synonyms for "outcomes" are "perfor- 
mance," "efficiency," "effectiveness," and "goals 

^and objectives" (that have been achieved). A 
survey of the literature on program evaluation 
and policy analysis clearly indicated that up until 
the present 'time there has been no generally 
accepted concept that serves to define the facts 

" known as "outcomes" and to discriminate among 
the potentially different types or classes of 
outcome measures.^ Widespread agreement on 
the need for some unifying concept has not been 
sufficient to overcome the theoretical complexity 

^ MMSures, AS used here, refer to concrete or quantifiable Indications of the 
presence and the extent, quality, or size of an outcome, for example, posttest 
versus pretest differences (where controls have been made for Input level) on a 
skill or knowledge achievement test, number of students being accepted by 
graduate schools, or score on a self-rating scale of perceived impact on 
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that characterizes the issue (Barton, t$61; 
Easton, 1965; Goodman, 1971; Schalock et al, 
1972; Micekand Wallhaus, 1973). 

Previous attempts to develop a systematic and 
useful conceptueiization of outcomes have been 
frustrated for a variety of reasons. Probably the 
most fundamental is the fact that postsecondary 
education (and education in general) performs 
multiple functions and generates a wide range of 
intended as well as unintended consequences. 
Some results (such as the number and type of 
degrees granted) are tangible enough to^ permit 
their identification and measurement without 
great methodological difficulty. Others are much 
more intangible and not readily susceptible to 
comprehensive, empirical investigation. In addi- 
tion, educatiohal programs areoft^n designed not 
so much for the production of immediate benefits 
as for the generation of the future, long-term 
effects. Educational outcomes are expected to 
occur over a wide range of time. These and other 
complexities have presented formidable barriers 
to attempts by educational researchers and 
practitioners to develop an "operational definition" 
of outcomes. -> 

The conceptual framework deWoped for the 
project has two key purposes: (1 ) identification of 
specific outcomes, and (2) understanding of their 
composition and characteristics. Identification is 
served through determination of whether or not a 
given entity actually constitutes an outcome. Jhe 
conceptual -framework specifies a set of 
characteristics or attributes common across 
outcomes. Understanding of the real nature or 
makeup of an outcome in the educational context, 
on the other hand, is accomplished through 
specification of the details of these attributes as 
they relate specifically to educational outcomes. 

Six attributes of an "educational outcome" that 
are meant to provide answers to the question 

c" 

achievement. Because measures of educational outcomes are often inexact 
find only superliclally Indicative of the outcome under consideration (as 
contrasted to the more dirsct. exact, and standardized implication of the 
generic definition of "measure"), some people prefer the term "Indicator." 
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''What are the characteristics and makeup of an 
educational outcome?'' are discussed in the first 
section of this Chapter. These attributes have 
been labeled as follows: (1) form, (2) change 
status, (3) focus, (4) neutrality, (5) measurability, 
and (6) output/impact. Following this is a section 
where other factors important to an understanding 
of educational outcomes are^discussed. Included 
are discussions of the following: Which 
institutional resources and activities are com- 
bined, and in which ways, to bring about the 
outcome(s) of concern? For whom (persons, 
groups, communities, or other entities) is the 
outcome intended, or who actually received or 
was affected by it? lV/7y will, or did, the outcome 
occur? Where will, or did, the outcome occur? 
IVften will, or did, the outcome occur? 

Several of the attributes and factors served as a 
basis for the dimensions included in the 
Outcomes Structure that are described in Chapter 
3. Others have implications for using the 
Outcomes Structure (for example, in developing 
lists of outcomes for different cells, of the 
Structure) and in analyzing outcome information. 
The relationship of the Outcomes Structure to the 
attributes and factors outlined will^be discu|sed 
in the sections that follow. 



seems clear that the three classes Schalock 
and his associates identified are applicable to 
the consequences of direct educational 
outputs as well as the outputs themselves. 
Furthermore, ^*form'' is probably a more 
descriptive term than either "structure" or 
"content" of what the three classes portray. 
Therefore, "form" was included as an attribute 
of "educational outcome," and the three 
classes are "product," "event," and "condi- 
tion." These classes can be especially useful 
when applied to the categories of the 
Outcomes Structure described in Chapter 3. 
Their main utility is perceived to be In building 
up lists of specific outcome variables and 
indicators for any category of the Structure 
that is oif concern, and in an analysis of 
outcomes. The three classes of "form" are 
defined as follows: 

a. Proc/ucf— tangible, concrete entities that 
endure with time, for example, a program 
completer, a degree, a job, a book. 

b. Even/— observable, tangible transactions or 
sets of behaviors that do not endure with 
time, such as a seminar, a concert, a 
graduation exercise, being listed in Who's 
Who in America, 



The Attributes of an "Educational Outcome" 

A total of six attributes of an "educational • 
outcome"— form, changestatus, focus, neutrality, 
measurability, and output/impact— help to 
answer the question ''What are the characteristics 
and makeup of an educational outcome?" 

1. Form. This attribute of an outcome refers to 
the makeup or substance of the outcome— thb 
forms in which particular direct outcomes of 
postsecondary e|^cation, or consequences 
associated with those direct outcomes, are (or 
are intended to be) observed and/or measured. 
In developing a classification of the direct 
outputs of educational research and develop- 
ment, Schalock and his associates (1972) 
empirically identified a dimension they called 
"structure" that outlined the basic makeup or 
the output. This dimension had three classes 
called "product," "event," and "condition." An 
earlier exploration by Schalock (1972), this 
time of student outcomes, determined a 
dimension called "content," that had three 
classes similar to those for ''structure": 
"objectives," "events," and "processes." It 



c. Co/7d/f/on— intangible but real circum- 
stances—morale, satisfaction, an attitude 
or belief, an appreciation, social equality, 
achievement, and so forth. 

2. Change Status. A second attrfbute of 
"outcome" is "change status." This attribute 
was suggested by the extensive work of Derr 
(1973), who developed a taxonomy of the 
"social functions" of education that had such a 
concept as its foundation. Derr was able to 
classify the ideas of each of the major 
educational theorists who has dealt during this 
century with educational purposes, into the 
categories and subcategories of his taxonomy. 
Upon inspection, it seemed clear to the 
NCMEMS staff that Derr's categories could be 
us^d to classify the functions of education for 
individuals as well ^s for society: the 
maintenance functions of stabilization or 
reproduction (for example, preserving, the 
culture has always been considered a primary 
function of education by many) and, the 
improvement functions of modification or 
replacement. Parsons (1951) referred to these 
same two broad functions as "maintenance" 
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and "adaptation." All educational goals are 
designed to preserve, replenish, reproduce, or 
stabilize the status quo, or to modify, enrich, 
restructure, revise, or replace what is current. 
Therefore, all educational outcomes can 
conceivably be thought about in these terms, 
and this realization led to the selection for the 
current project of "change status" as a primary 
attribute of "educational outcome." 

A problem with the term "improvement" used 
by Derr and the term "adaptation" used by 
Parsons is that they^have value connotations 
suggesting that the outcomes in these classes 
are viewed as being positive. Outcomes often 
will be viewed as desirable by some people and 
undesirable by others. Therefore, a neutral 
term should be used for this category, if 
possible. After much deliberation, the 
following, somewhat neutral, terms were 
selected for the two major classes of "change 
status," arid included as "fourth level of detail" 
categories"^ of the "type" dimension in the 
Outcomes Structure:. ' 

a. Maintenance—outcomes that result in 
l<eeping the status quo: in stabilization or in 
reproduction and preservation— for ex- 
ample, continuing traditions into the next 

^generation, helping a.student to keep from 
becoming "rusty" on basic academic skills, 
or recording current knowledge in books 
and other documents so that it will be 
preserved for future use. 

b. Cftange— outcomes that result in alteration 
of the status quo: in modification, revision 
(improvement or otherwise), or replacement. 

The traditional purposes of a postsecondary 
education have often been couched in terms of 
maintenance or change. Furthermore, these 
two outcome directions require decidedly 
different orientations and strategies. As a 
result, the "change . status" attribute was 
considered to warrant inclusion as a part of the 
type-of-outcome dimension of the NCHEMS 
Outcomes Structure: 

3. Focus. "Focus" is similar in concept to the 
"form" attribute discussed previously but has a 
different emphasis. Dictionariesdefine "focus'* 
as a' point of concentration and this attribute 
describes the point of concentration— the 
basic, specific '^what"— that is maintained or 
changed to constitute* the outcome of concern. 



(Another appropriate name for this attribute 
would have been "aspect," as used by the 
Swedish LIGRU taxonomy of educational 
objectives [Klingberg, 1970].) To Illustrate, 
instruction and socialization can involve 
maintenance or change on such entities as 
knowledge and understanding, skills and 
competencies, attitudes and values, apprecia- 
tions, habits, roles, reputation, certification 
and licensure, status, jobs, income, economic 
security, family relations, social interactions, 
social conditions, community standard of 
living, and gross national product. Research 
and scholarship involve maintenance or 
change on such entities as basic Knowledge 
and understanding, applied knowledge and 
understanding, techniques and technology, art 
forms, and the basic entities within people and 
communities which these impact. Public 
. service involves maintenance and change on 
theservices that constitute its outputs, and on 
the basic entities these services impact. 

Most of the taxonomies and classifications of 
educational outcomes and objectives so far 
developed^^have included "focus" types of 
categories. The forerunner of the NCHEMS 
Outcomes Structure, the NCHEMS Inventory of 
Higher Education Outcome Variables and 
Measures, is essentially this kind of a 
taxonomy. The "focus" attribute was used 
along with "change status" in developing 
categories for the type-of-outcome dirnension 
of the Outcomes Structure presented in 
Chapters. 

4. Neutrality, A fourth attribute of "educational . 
outcome*' Ms one that can be called 
"neutrality." In the past, some people have 
more or less equated "educational outcomes" 
to "educational benefits." Such an approach, 
however, tends to direct studies of outcomes 
only to the positive aspects of education, 
neglecting the potentially undesirable results 
of education that could have just as much, if \ 
not more, impact or effect on individuals, 
families, and society as dp many of the 
potentially desirable results of education.. 
Clearly, outcomes viewed as having negative 
value or costs shouid be considered just as 
carefully in postsecondary education planning 
and management as those outcomes viewed as 
having positive value. 

There are many consequences of education 
generally thought to be desirable by most 
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people. Examples Include student adaptability, 
a nnore knowledgeable and skilled population, 
increased economic status of college 
graduates, improved health care due to 
research advances, and an increased commu- 
nity standard of living. Hov/ever, there are also 
presumed consequences generally thought to 
be viewed In negative terms by the majority of 
the country's population. For example, Bowen 
(1974) lists drug use, political radicalism, and 
religious dissent as impacts generally viewed 
in negative terms. Although most people In 
society may view a particular educational 
impact as being a^>9sltlve or negative 
consequence, however^ there will almost 
always be a minority that sees It In the- 
opposite way. And the population may be quite 
divided In its view of certain presumed impacts 
of college, such as the impacts of university 
laboratory discoveries on population growth or 
on ecology. The majority of the population may 

. feel neutral or noncommittal about other 

- outcomes. 

The point being emphasized here, which Is 
sometimes overlooked by educators as well as 
others, is that the generic concept of 
Veducallonal outcome*' Is a neutral one 
' separated from any Inherent value status. 
Planners and managers need to hold to this 
concept of "outcom^6" as a neutral term so that 
values do not cause them to Ignore potentially 
negative or unexpected Impacts In their 
'assessment of needs, goal-setting, program 
evaluation, and so forth. (This Is a primary 
rationale given in support of Scriven*s "goal- 
free" evaluation.) It. Is important to keep in 
mind, however, that people do attach value 
connotations to the particular outcome and 
make It into a value-oriented concept. The 
same outcome may be viewed as a benefit by 
some and as a detriment by others, while still 
others take a neutral position In the matter. 

5. Measurability. "Measurablllty" is an additional 
attribute that helps to give Insight Into the 
"what" question. Somfe people claim that any 
outcome can be measured, while others 
suppose that certain outcomes cannot be 
measured— but this attribute does not concern 
itself with such a distinction. Measurability, as 
used here, refers to the extent and ease with 
which a particular outcome can be quantified. 
It Is related to the tangibility, or concreteness, 
that was one of the two characteristics (the 
other one was endurance) that helped 



differentiate the product, event, and condition 
categories given for the "form" attribute 
discussed previously. However, It Is more than 
this. Some rather abstract and .Intangible 
outcomes can be measured quite readily In 
quantifiable terms, for example, mechanical 
aptitude, reading comprehension, and voca- 
tional readiness. 

Some outcomes are ea&y to measure; others 
are more difficult to measure. One who has 
especially focused on this distinction is Gross 
(1973), although he talked in terms of goals 
Instead of outcomes. His breakdown of 
easy-to-measure versus hard-to-measure goals 
for different target populations Is shown In 
Figure 2.1. 

Although knowledge about the measurability 
of particular categories of outcomes is 
extremely Important to outcomes identifica- 
tion, analysis, and Interpretation — It Is 
important to know which categories of^ 
outcomes are difficult to quantify and which 
are easy to quantify— this attribute was. 
rejected as a dimension of the Outcomes 
Structure. First, the boundary between the two 
categories (or the three categories If an 
in-between one Is added) could not be made 
distinct enough that different people could 
classify outcomes near the boundary. Finally, 
what one measurement expert may consider as 
,easy to measure,^ based on the availability of a 
particular measure, may be viewed as drfflcult 
to measure by another measurement expert 
who considers that measure to^be hvalld. (For 
example, some measurement experts would 
consider scores on a reliable self-report 
instrument to be a valid measure of a criterion; 
others would consider the scores invalid.) 
These problems do not, however, negate the 
, Importance of trying to determine; from one's 
own perspective, the measurability of 
outcomes described in the various sections of 
the Outcomes Structure of Chapters. 

6. Output/Impact. Another attribute that can add 
to the the understanding of "what Is an 
outcome" is the "output/Impact" distinction. 
Many outcomes researchers have focused on 
outputs as educational outcomes, that Is, the 
conditions, goods, and ^services that --the 
programs and institutions of postsecondary 
education produce as a direct transformation 
of the input elements. A few researchers in the 
outcomes area, have focused on impacts as 
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Figure 2.1 

GROSS'S PRESENTATION OF "EASY-tO-MEASURE" AND "HARD-TO-MEASURE" GOALS 



Benefits To 


Goals Easily Operationalized 


Goals Hard to Measure 


^^t; 

1. Society 

2. Individuals 

3. Employers 

4. Government 

5. Institutions 


1 . Improved equity (income, employment) 

2. Increased GNP 

3. Reduced unemployment 

4. Increased social satisfaction 

a. social institutions 

b. job satisfaction 
^ c. overall satisfaction 

1. Increased incomes 

2. Reduced unemployment 
Increased satisfaction with 

a. work > 

b. general conditions 

c. social status 

1. Jobs of specific employers filled 

2. Jobs in particular areas filled 

1 . Increased tax revenues through increased 
tax base 

2. Increased numbers of qualified persons for 

1 . Meet the need for quality undergraduate and 
graduate level output 

2. Improvi equity (income and educational 
. opportunity) 

3. Improve the level of human capital for 
industry, agriculture, business, government, 
etc. 

4. Meet community adult education and continuing 
education needs 


1 . Reduced asocial behavior 

2. Reduced dependancy on government 

3. Improved family life 

4. Improved race relations 

5. Improved health 

6. Improved housing 

1. Reduced dependency 

2. Improved health 

3. Improved family life 

4. Improved housing ^ 

1 . Increased productivity of work force in 
particular parts of labor force 

1 . Reduced cost of government questions 
(health, welfare, law enforcement, etc.) 

1 . Improve levels and sensitivities in community 

2. Improve chance of individuals reaching higher 
levels of self-fulfillment and competence 

3. To advance knowledge through 

a. organization of leiarning 

b. research and publication 



From "A Critical Review of Some Basic Considerations in Postsecondary Education Evaluation" by -P.F. Gross, Po//cy Sciences Vol. 4, 
No. 2 (1973), page 186. Used with permission of the Elsevier Scientific Publishing Company, Amsterdam. 



outcomes, that is, on the consequences of the 
outputs. Other investigators have concurrently 
examined outcomes in terms of both outputs 
and impacts, in doing so, however, the 
concept of "output" has often been used as 
though synonymous or combined with the 
term "impact." Educational outputs have not, 
in these cases, been cieariy distinguished from 
educational impacts. 

The failure to make a conceptual distinction 
between outputs and impacts reduces the 
ability to identify and organize systematically 
the wide range of educatipnal outcomes, and 
also reduces the ability to analyze educational 
outcomes. Both concepts are crucial but it is 
important that they remain distinguished irom 
one another. Dye (1975), for example, has 
observed: 2 

Policy impact is not the same as policy 
output. It is not enoltgh to measure 
benefits [for instance] in terms of 
government activity. For example, the 
number of dollars spent per member of a 
target group (per pupil educational 
expenditures, per capita welfare expendi- 
tures, per capita health expenditures) is 
not reaUy a measure of the impact of a 
policy on the group. It Is merely a measure 
of government activity— that is to say, a 
measure of policy output. We cannot be 
content with measuring how many times a 
bird flaps its wings, we must assess how 
far the bird has flown. In describing public 
policy, or even in explaining ^its determi- 
nants, measures of policy output are 
Infiportant. But in assessing the impact of 
policy, we must first identify changes in 
the environment . . . that are associated 
with measures of government, [p. 328] 

These observati^jis by Dye about a political 
system are equally applicable to The 
educational system. ^ For the study of 
educational outcomes, rt is important to 
distinguish conceptually measures of educa- 
tional. outputs from measures of educational 
impacts without losing sight of either set of 
measures. Dye and other policy analysts 
(Easton, 1965; Robinson and Majak, 1967; 
Cook and Scioli, 1972) have in effect implied 
that the outcomes analyse in postsecondary 
education needs to seek separate answers to 
two questions: 

1. vyhat direct products, events, and condi- 



' tions (outputs) do institutions of post- 
secondary education produce? 

2. What differences (impacts) in the real world 
result from these products, events, and 
conditions— what other products, events, 
and conditions result? 

Educational outputs have been defined by 
some as the "first-order consequences" of 
institutional and programimatic activities 
because they can typically be direcxiy linked, at 
least in concept, to^ those activities. 
Conversely, educational impacts are defined as 
"second-order consequences" because the 
linkages to the institutional and programmatic 
activities are indirect. 

Institutional or program planners will probably 
consider particular second-order consequences 
(impacts) in their planning, although they may 
never suspect other such consequences that 
will result (because they were never 
considered or were thought to be impossible, 
as well as not being intended or aimed for). 
Because educational impacts are related to .the 
activities in an indirect manner, hoWever, the 
institutions and programs may have little or no 
control over such impacts. For instance, 
several researchers have found that college 
education is highly correlated ta the level of 
one's earnings, implying that, the college 
education has an impact oh one's ability to 
earn. IHowever, it is difficult to demonstrate 
that one's earning ability should be attributed 
primarily to college education. For both the 
college educated and for others, the ability to 
earn is to a large regree dependent upon^ 
prevailing m'arket conditions and other factors: 
Educational institutions obviously have no 
direct or immediate^ control over such 
conditions. 

Another point that should be mentioned Is the 
relationship of the "output/impact" attribute to 
the time when the outcome occurs, a factor 
discusssed in the next section. Because of its 
direct linkage to the institutional or program 
activity or activities that caused it, an output 
must appear or take shape during or directly at 
the end of the procecs activities that caused it. 
An impact, on the other hand, is less 
immediate than the output (or as is more likely, 
the many outputs) that initiated it, and it also 
is less immediate than any chains of 
intermediate impacts leading to it. An impact 
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can occur during the process (such as citizen 
enjoyment [impact] of a concert [output] 
sponsored by the rnstitution), or at any time 
thereafter (even many years later [such as more 
enjoyable retirement"" years because of 
attending college-sponsored concerts when a 
student]). 

Since the primary factor distinguishing 
outputs from impacts is whether or not there is 
a direct link between the outcome and the 
msiituticnal activities respoj}s\b\eJ!9m that 
outcome occuring ("immedi^" and **amount 
of control" are secondary factors), we have 
defined "outputs" and "impacts" as follows: 

a. Oufpufs— the direct end Jv^lwc*^* events, 
or conditions that result from the 
application of the institutional or program 
processes to transform the various inputs. 
Examples for institutions are achievement 
levels, specialization of knowledge, de- 
grees, program completers, publications, 
cultural or entertainment events. 

b. /mpacfs— the consequences of outputs and 
earlier impacts (the indirect products, 

.events, or conditions produced) for 
particular individuals, communities, or 
things. -''Impacts" answer the question of 
what differences result because of outputs 
and earlier impacts. Examples of possible 
impacts for -institutions include a -program 

. completer's ability to obtain and hold a job, 
the security and income or the prestige the 
job gives a person, the increased gross 
national product that results from increased 
income of individuals, the* increased 
standard of living and quality of life that 
may be associated with incf-eased gross 
national product, and so on. 



Because >of their direct relationship to the 
process activities causing them, outputs are- 
easier to research than are impacts (although 
there are serious research problems even for 
them). Therefore, most of the research done on 
educational outcomes has focused on outputs. 
However, in spite of their indirect nature, 
which makes it especially difficult to show 
cause and effect relationships, impacts have 
been the focus of a number of empirical 
investigations. f Most of those studies have 

^ Most notably these studies have been in the areas of human capital iSchuttz. 
1963. Becker. 1964). behavioral Influences (Feldman and Newcomb. 1969. 
Pace. 1972). attainment of selected spciat objectives (Krathwohl and Payne. 
1971). and intellectual accomplishment (Schalock. 1972). 
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focused on the impacts on. individual students 
and graduates and .include only minimal 
treatment of postsecondary impacts on the 
community and society at large. On the other 
hand, it must bo kept in mind that impacts at 
the societal level are aggregates of the impacts 
on individuals. It is an extremely compl^^ 
aggregation, however, because it involves 
concepts such as the "multiplier effect." 

This attribute makes an extremely important 
distinction that can be useful in identifying and 
generating lists of outcomes and in analyzing 
outcomes. It was not used as a dimension of 
the NCHEMS Outcomes Structure, however, 
because some outcomes would be considered 
an output by particular people and an impact 
by others. A skill such as welding can be used 
as an example. Some of those using the 
institution , the program, or the course as a unit 
of analysis (the causative unit on which 
attention is focused) would classify this as an 
output because they would contend that it is a 
skill provided directly by the unit of concern. 
Others using the same unit of analysis would 
content that the component skills of welding 
must be mastered, prior to the student having 
the "skill of welding-^' which would mean that 
the "skill of welding** has an indirect 
relationship to the activities causing it, 
through the component skills of welding. From 
their viewpoint, the component skills of 
welding become educational outputs that 
result in a skill impact, the overall "skill of 
welding." The same rationale could be used for 
any developmental outcome that has p^'^requi- 
site components. In addition, for many 
outcomes there may be contributioos to^ their 
existence from particular outputs and their 
impacts.that add to the major contribution that 
comes directly from the producer or producers 
of these outcomes, in which case the outcome 
might be considered as part output and part 
impact. 



Other Factors important to Understanding ^ 
"Educational Outcomes" 

In addition to the attributes of "educational 
outcome," a total of five other factors are 
important to an understanding of this concept. 
These factors have been labeled producer/ 
facilitator, audience, intended/unintended, func- 
tional area, and time. Each factor relates directly 
to one of the following questions (which were 
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noted at the beginning of this chapter), and ip the 
order listed: . Which institutional resources and 
activities are combined, and in which ways, to 
bring about.the outcome{s) of concern? For whom 
is the outcome intended, or who actualiy fBcelved 
or was affected by it? Why^ wiil, or did, the 
outcome occur? Where wili, or did, the outcome 
occur? When wili, or did, the outcome occur? 

• Producer/Facilitator. Educational outcomes 
generaiiydo not just happen; they are caused or 
influenced to happen and conditions are such 
that they are allowed to happen. Even 
unintended outcomes are typically stimulated 
orcaO^od to happen by a producer/facilitator or 
combination of producers/facilitators. Similar- 
ly» accidental outcomes usually do not just 
happen. Some actions or activities bring them 
about. 

It is important to recognize that the 
phenomenon called an '/outcome" is a relative 
thing. An outcome from' one point of view may 
well be seen as an input from another 
perspective. To illustrate, for a college 
"graduates produced** constitutes an outcome, 
while for business firms these graduates are 
considered inputs. Thus it is necessary to link 
outcomes to the unit or entity that produces 
them in order to maintain a consistent 
perspective. Since postsecondary institutions 
and their components have the basic 
responsibility for generating postsecondary 
education outcomes, the institution and its 
programs are the focus usect in this 
conceptualization of outcomes.^ 

Traditionally, the programmatic or functional 
activities'Of an educational institution and its 
components that produce and facilitate (or are 
intended to produce and facilitate) particular 
outcomes have been divided into three 
functional areas: instructional and socializaf^n 
activities, research and scholarship activities, 
- and public service activities. Instruction and 
socialization are the formal (curricular) and 
informal activities provided to help bring about 
student growth and devejopment, that is, 
knowledge, understanding, competencies, atti- 
tudes, appreciations, habits, and so forth. Over 
the years, a wide variety of programs and 
methodologies have been tried in different 

^The focu$ on institutions and their component pfograms (as the "unit of 
analysis") also applies at the more composite levels of educational planning, 
for example, the use of outcomes information for educational decision making 
and policy development at (he state ano federal level. 



settings to stimulate student growth and 
development. 

Research and scholarship^' activities are 
conducted by units or individual staff mem>>ers 
within the institution with the aim to dvwelop 
new knowledge or art forms. The new 
knowledge, techniques, or forme of expression 
can be designed, either to have practical 
applidation (forexample, "applied research") or 
merely to be new (for example, Vpure 
research"). 

Public service activities are those activities that 
aim to benefit directly or have an impact on the 
communities or groups of individuals residing 
within the service area of the institution. Many 
public service activities are instructional in 
nature, such as extension courses and other 
community education services.. Others are 

>. advisory or entertainment, sUch as extension 
advisory and consulting services for individuals 
and businesses, community problem analysis 

"^ei^vices, and cultural and entertainment events 
or facilities for thfe community. 

It should be noted that these three traditional 
focuses of postsecondary education institu- 
tions correspond to the primary programs of the * 
NCHEf^S Program Cfassification Structure 
(PCS), outlined in Figure 2.2. Similarly, the six * 
support programs of ttie PCS (academic sup- 
port, student service, institutional administra- 
tion, physical plant operations, student 
financial support, and independent operations), 
could be expected to also generate educational 
outcomes. 

* The producer/facilitator dimension is critical in 
any attempt to identify, classify, .or analyze 
outcomes ibecause different types or levels of 
programs and organizational units are designed 
to prpduce particular kinds of outputs and 
impacts. For example, many of the outcomes 
intended for an introductory biology course 
may be different from those intended for an 
advanced biology course, for a degree-oriented 

" program in the biological sciences, for' a 
biology department, or for the institution as a 
whole. 

One major probiem*^ with the producer/ 
facilitator dimension is referred to as the 
' "jointness problem,** where two or more types 
ofr progranris or other organizational units may 
have contributed to the production of^ 
particular outcome, and it is dijficiJlt to 
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ascertain their relative contributions (it may 
even be impossible to ascertain this). 
Therefore, a particular outcome can and often 
should be associated with tnore than one 
category of producer/facilitator— which sug- 
gests that "producer/facilitator" would not 
make a good formal dimension fortheNCHEMS 
Outcomes Structure. 

Another problem with using producer/facilita- 
tor as a dimension of the Outcomes Structure is 
that it is difficult to determine which producer/ 
facilitator units actually contributed to the 
production of an outcome. The jointness 
problem is one reason: Another reason is that 
the environment of the institution, program, ot 
other UQit (such as student. peers, atmosphere, 
and reputation) affects the outcomes produced 
by interaction with the producer/facilitator 
components. Similarly, a wide variety of 
methods, techniques, and tools can interact to 
constitutie the process within the producer/ 
facilitator unit of concern; and each 
combination might be expected to result In 
different educational outcomes. In addition, the 
.input characteristics of the entities being acted 
on— such as the abilities of the incoming 
students— will make a difference in, the 
outcomes to be expected and In the outcomes 
acjually attained. 

It is especially difficult to relate actual 
outcomes attamed to specific program units 
because of the number of. affecting variables 
and the complexity of the Interactions, but it is 
still important to realize that there are such 
producers/facilitators. Environmental and cli- 
mate variables, direct process variables, and 
input variables all affect the outcomes. 

It should be kept in mind that there are other 
ways that producers/facilitators of outcomes 
can be categorized that could be just as useful 
in analyzing and evaluating outcomes. For 
example, instead of focusing on activities 
grouped according to the program categories of 
, an organizational framework like the NCHEMS 
Program Classification Structure, one could 
focus on strictly purposive groups of activities, 
such as activities aimed at development of 
general or specific knowledge, development of 
competencies and skills, development of new 
knowledge and art forms, providing analytic 
and advisory assistance. Similarly, one could 
focus strictly on process— on types of 



activities, such as lecturing, discussion, 
investigation, preparing reports and news- 
letters, consulting. 

• Audience, "Audience" refers to tffe persons, 
groups, organizations,* communities, or other 
entities that receive or are affected by, or that 
are meant to receive or be affected by, the 
educational outcome(s) of concern. While on 
the surface this idea may seem straightforward, 
it actually presents one of the major difficulties 
in identifying and understanding educational 
outcomes. These difficulties result from the 
great complexity that characterizes the 
individuals, groups, communities, and other 
populations who directly or indirectly are served 
or affectejd (or are meant to be served or 
affected) by Ihe outcbmesTof postsecondary 
education. According to Gross (1966), the 
performance of a^ny social system consists of 
activities designed "to satisfy the interests of 
vjarious 'interesteds"' (p. 184). An example of 
these various "interesteds" is provided in 
Figure 2.3, which is a reproduction of a table 
prepared by Gross (1966, p. 173), to illustrate 
the variety and complexity of social systems 
potentially affected by the actions of 
organizations such as colleges, universities, 
and other postsecondary education institu- 
tions. In addition to such "interesteds,'' 
particular institutional and program outcomes 
may be primarily aimed at and/or received by 
subpopulations of people havjng particular 
observable characteristics, such as retired 
persons, homemakers, women, blacks, and the 
verbally handicapped. 



Because in planning, management, and 
evaluation it is'cfucial to keep straight who the' 
outcome is^aimed at, and because audience 
categories can be created that have little 
overlap, the "audience" factor was incorporated 
into the Outcomes Structure as one of the three 
formal dimensions. As outlined in Chapter 3, a 
number" of different categories and subcate- 
gories were formulated for the "audience" 
dimension of the Structure (for a definition of 
any of the following categories, see page 24): 
(1) Individual/Group C//enfs— students, former 
students, family and relatives of the students 
and former students, peers and associates of 
students and former students, faculty, staff 
other than faculty, and other individual/group 
clients; (2) Interest-Based Communities-- 
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Figure 2.3 
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♦These columns include only simple systems. Complex systems are networks composed of formal organizations (usually di^erent types), 
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f As here, defined, "territorial entity" includes a variety of other social systems within its spatial boundaries. Almost every territorial entity 
a complex system. 
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private enterprise communities, association 
communities, government communities, non- 
govemmental/pubiic service communities other 
than the institution producfng the outcome, the 
institution or institutionai unit that caused the 
outcome to occur, and other interest- 
based communities; (3) Geographic-Based 
Commun/Y/es— iocai community,' the state, a 
region, the nation, an internationai community, 
and other geographiC:based communities; 
(4) Aggregates 0/ Peop/e— subpopulations 
based on abiiity ievei, age, educational ievei, 
occupation, physical disability condition, race, 
sex, and other factors. 

* 

• Intended/Unintended. The "intended/uninten- 
ded" factor helps to ansvyer the "why" question. 
Outcomes are generally either "intended to 
occur". (in which case the producer/facilitator 
attaches a purpose or purposes to an outcome) 
or "not intended to occur" (in which case no 
purposes are attached to the outcome). Many of 
the negatively viewed educational outcomes 
that occur are not expected by those planning 
the educational activity that causes or 
facilitates them. They are **Unintended 
outcomes," which may occur instead of the 
intended outcomes or along with the intended 
outcomes. Unintended outcomes are often 
referred to as "side effects," and particular side 
effects are of ten unexpected, but they may al3b 



be expected. Sometimes previous experience or 
research may suggest that negative side effects 
will occur but the planner considers the 
benefits of the intended outcomes to outweigh 
the negative side-effects Enough to warrant 
proceeding with the activity or program. 

It should not be implied from the preceding 
discussion, and from the fact that intended 
outcomes are almo3t always viewed as being 
desirable by the educational planner, that the 
unintended outcomes are always or usually, 
negative outcomes. A majority of side effects 
may be of positive value for some programs or 
activities. 

Concerning intended outcomes, a major reason 
institutions and programs provide outcomes is 
because they will receive something of value in 
return. The focus on institutions and their 
component programs as the basic unit of 
analysis assumes that institutions supply 
educational goods and services because these 
are desired and demanded by various members 
of ,the society (or at least the institutions 
perceive a demand for these goods and 
services). In exchange, society provides 
financial and other necessary resources, plus 
perhaps nonpecuniary returns such as status 
and praise. Figure 2.4 depicts this educational 
exchange system. 



Figure 2.4 
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To produce educational goods and services 
efficienlly and also to attract sufficient 
resources, institutions need to know the quality 
and types of educational goods and services 
Jhat are- produced. They also need to 
understand the costs, and what impacts these 
goods and Nservices have on members of 
society. 

Some organization theorists would stress that 
the major outcomes toward which all 
organizational entities are striving are survival, 
adaptation, and growth, with all other goals 
being subsumed under these three basic ones. 
Institutions and the people in those 
institutions, on the other hand, would stress 
altruistic purposes such as aiding the local 
community and its residents in particular ways, 
and exhibiting a concern for others and the 
welfare of- society. In either case, the exchange 
system is operating. In exchange for helping 
the comnriunity, the members qf th§^institutioh 
may receive altruistic satisfaction plus 
resources that will allow them to continue their 
assistance. 

This discussion of the intended/unintended 
factor emphasize^ two things. The first is that 
important unintended outcomes of post- 
secondary institutions and their programs do 
occur, and that the possibility of such 
outcomes needs to be considered in the 
planning and management process taking place 
within institutions. The second is that 
expressed purposes (plus perhaps some 
unexpressed purposes) will always be attached 
to intended outcomes by producers/facilitators, 
and that it is important to consider such 
purposes in planning for and analyzing 
educational outcomes. 

• Functional Area. The "audience" factor, related 
to the "for whom" question, also says 
something about ''where" the outcome is 
occurring (in^ individuals, in their homes, in 
their neighborhoods, and m a variety of 
communities). However, "where" also refers to 
functional areas within those various entities. 
Parsons (1951) formulated four functional or 
behavioral areas that apply to both individuals/ 
groups and communities: (1) politics, (2) econ- 
omy, (3) integration, and (4) maintenance. The 
life of any individual; group, institutior or 
community involves similar functional areas. A 
potentially useful formulation for presenting a 



picture'T>f educational outcomes is to expand 
on the politics and economy categories so that 
each major functional aspect affected by 
postsecondary education is covered. One 
possible breakdownitiat may have potential for 
this purpose is: (1) economic outcomes, (2) edu- 
cational/technological outcomes, (3) politicaL 
outcomes, and (4) social /cultural/personal 
outcomes. Examples of outcomes for each of 
these functional areas are shown in Figure 2.5. 

• Serious consideration was given to Including 
the four functional areas as categories for one 
dimension of the Outcomes Structure. 
Although the ter^ "function" ^s value laden, the 
antecedent terms are not; for example, the 
social/culturai/personal, economic, and polit- 
ical areas of focus in a person, group, or 
community can clearly be of either positive or 
negative value. Furthermore, the four areas 
have much meaning to educators <vhe 
traditional goals of education directly relate to 
them) and it is useful to think in such terms 
when analyzing outcomes. The problem, from 
the perspective of classifying outcomes, is that 
there is too much overlap among the classes. 
For example, most outcomes in the political 
area are essentially also outcomes in the social 
area, and developmental outcomes in almost 
every area could be considered outcomes in the 
educational area as well. (An "educational" 
category still would be needed, even though it 
could be considered to overlap all the other 
categories, to be able to classify such 
outcomes as degrees and program completers.) 
Because of this problem, a "functional area of 
occurrence" dimension was rejected for the 
Outcomes Structure. The concept did, however, 
influence the ^*type" dimension finally 
formulated; for example, note that it has 
classes such as "economic outcomes" and "art 
forms and works outcomes." 

• Time. Time is the "educatjonal outcome" factor 
that pertains to the *'when" question. This 
factor has two important aspects: (a) the 
point-in-time when the outcome occurs, and 

, - (b) how long the outcome lasts. 

As described earlier, time is one of the 
secondary factors that distinguishes outputs 
from impacts. By definition, outputs occur only 
during or at the end of an institutional or 
institutional-unit process, while an impact can 
only occur after the output(s) that led to it, 
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Figure 2-5 - . 
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The societal outcomes listed here have been hypothesized by different people as being impacts of postsecondary education on our 
society— note that some would beconsldered to be of positive value by most people while others would be considered detrimental by 
many people. As Indicated earlier, the extremely complex Interactions continually taking place in society make it very difficult, and 
maybe Impossible, to isolate ihe societal impacts of postsecondary education from the Impacts of other social institutions and 
student experiences outside of postsecondary education. There Is definite empirical evidence lr>oicatlng that postsecondary education 
has a significant impact on many of these "outcomes." while others seem to be strictly conjecture. 
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although at any point after the output comes 
intp being. The time when the outcome occurs 
is especially important for impacts because any 
educational impact can be expected to lead to 
other Impacts. For analysis It Is helpful to know- 
when (in relation to the process) and where In 
the chain of intermediary Impacts a particular 
impact fits. Lenning (1974) has combined "time 
when the outcome occurs" with "audience" and 
presented this causal chain as a pyramid where 
student outcomes constitute the base, 
postgraduate outcomes constitute the mid- 
section, and societal outcomes constitute* the 
apex. The pyramid Illustrates the fact that many 
societal outcomes of postsecondary education 
depend largely on students* postgraduate 
otitcomes to bring them about, which in turn 
depend 0.1 the outcomes when they were 
students. 



might be considered "long duration." Figure 2.6 
gives examples of short-duration and long- 
duration outcomes for both outputs and 
impacts. 



Summary 

We ha;9 discussed six attributes orcharacterlstlcs 
of an "educational outcome" plus five other 
factors Important to an understanding of this 
concept. These attributes and factors are 
responses to a number of . questions atx)ut 
^outcomes. The questions, the attributes and 
factors associated with each, and what they mean 
are summarized below: 

, A. What are the characteristics and makeup of an 
"educational outcome"? 



The time, expected time, or time preference of 
occurrence of an outconrie In students* lives can 
have serious implications for data collection 
and analysis, for curriculum planning, and for 
other institutional decision making concerning 
students. Therefore Astin (1970) included "time 
of outcome" as one of the three dimensions in 
his taxonomy of student output measures, and 
Schwartz and Tiedeman (1957) Included "time** 
as. a dimension of their classification of 
educational objectives. But It seems impossible 
to specify particular categories of time that will 
satisfy most people*s cieedain classification of 
educational outcomes. Furthermore, time 
categories Important for planning student 
outcomes often will have little or no moaning 
for other audience categories. For these 
reasons "time" was at first eliminated from 
consideration as a dimension of the NCHEMS 
Outcomes Structure presented in Chapter 3. In 
the end, however, to emphasize Its strategic 
importance in the use of outcomes information, 
a "time" dimension was added that has no 
specifically defined categories (the categories 
to be applied are left up to the complete- 
discretion of the user of the Structure). 

Conceming how long the outcome lasts, an 
outcome'can be of short duration or it can be of 
long duration, ^where the dividing line between 
"short" and "long" is relative and depends on 
the situation and the perceptions of the viewer. 
For one person or situation, an outcome 
persisting from graduation until 1 year later 
might be considered "short duration." For 
another person or sftuation, such persistence 



• Form— the basic configuration of the out- 
come as it is observed and/or measured. 
Outcomes can be separated into products, 
events, and conditions. 

Change Sfafus— whether the outcome re- 
sults in maintaining (preserving, replen- 
ishing, reproducing, or stabilizing) or 
changing (modifying, enriching, restruc- 
turing, or replacing) theexisting.cpndition or 
state of affairs. 

Focus— the basic, specific "what" that is 
maintained or changed to constitute the 
outcome of cpncarn (knowledge, under- 
standing, skills, attitudes, roles, certifica- 
tion status. Jobs, income, social conditions, 
technology, art forms, and so forth). 

A/eu fra//fy— although people attach positive 
or negative value connotations tQ specific 
outcomes, the generic concept of "outcome" 
is a neutral one separated from any inherent 
value status. 

Measurability-^the ease witrt which the 
outcome can be q^ntffled or measured. 
Some outcomes are easily measured; others 
are difficult to measure. 

Oufpi/////77pacf— whether there is a direct 
link between the outcome and its producer/ 
facilitator (output), or an indirect linl< be- 
tween the outcome and its producer/facili- 
tator through outputs an1d intermediary 
impacts (impact). 
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Figure 2.6 

EXAMPLES OF SHORT-DURATION AJjID LONG-DURATION OUTPUTS AND IMPACTS 



Outputs Impacts 



S/70/t-Puraf/on— Lasting but a little time; of 
brief duration. (Where the dividing line is 
between short-term and long-term should be 
decided on and defined by the institution and 
will depend on its particular needs.) 

,\ 


A college footballgame. 

Placement in an entry-level 
position In the "occupational 
world." 


The (Bnjoyment by spectators*^ 
as they watched a college 
football Oame. ' 

On-the-job learning experi- 
ences during ihe first year 
after college to prepare for a 
second-level advancement. 


/.ony-Dt/raf/on— Lasting an extended or per- 
manent period of time; of great duration or 
longevity. (Where the dividing line is 
between short-term and long-term should be 
decided on and defined by the institution and 
will depend on its particular needs.) 


Development of a vaccine for 
cancer. 

Holding of a degree. 


Eradication of carreer from 
our society and thd resulting 
increase in life expectancy. 

Job security and greater 
lifetime earnings resulting 
from holding an earned 
degree. 



B. Which institutfona! resources and activities are 
combined, and in w/}/c/? ways, to bring about 
theoutcome(s) of concern? 

• Producer/ Facilitator— the programmatic or 
functional activities of an educational insti- 
tution or its components that produce and 
facilitate, or are intended to produce and 
facilitate, particular educational outcomes. 

C. For whom is the outcome interided, or who 
actually received or was affected by it? 

• Aucf/ence— the persons, groups, organiza- 
tions, communities, aggregations of people 
with common observable characteristics, 
activities or other entities that receive and /or 
are affected by (or are intended to receive or 
be affected by) the outcome of concern. 

D WftywiH, or did, the outcome occur?'' 

• Intended/Unintended— whether the outcome 
was designed or planned to occur or whether 
it Just happened. Included are the positive, 
negative, or neutral value connotations 
attached to an outcome by different people 
and groups, and the "exchange value" 
perceived for the outcome by its producer/ 
facilitator. 



E. lV/)ere will, or did, the outcome occur? 

• Functional Area— the fcinctional areas within 
the various audience entities that are being 
affected by (or that are meant to be affected 
by) the outcome, such as economic, educa- 
tional/technological, political, and social/ 
cultural/personal. 

F. IV/yen will, or did^ the outcome occur? 

• Time— the tirfie, or expected lime, of occur- 
rence of an outcome (such as> prior to gradO- 
ation, more than one year after graduation) 
and the duration or persistence ;^of the 
outcome (how long it lasts). 

Several of these attributes and factors served as a 
basis for dimensions of the OutcomeS|Structure 
described, in Chapter 3: audience, change status, 
focus, <and time. Others have implications for 
using the Outcomes Structure (for example, in 
developing lists of outcomes for different cells of 
the Structure) and in' analysis of outcomes 
information.. 
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Chapter 3 

A PROPOSED OUTCOMES STRUCTURE 
FOR PdSTSECONDARY EDUCATION 



Introduction 

By definition, a classification system (or 
structure) for outcomes consists of one or more 
dimensions divided into associated categories 
and subcategories for organizing, differentiating, 
and showing relationships among outcomes.? A 
large number of varied attempts have been made 
using many different dimensions to formulate 
classification systems for educational outcomes 
and such outcome-related concepts as goals and 
objectives. A list of over eighty such , 
classifications found by Lenning (1977) is in 
Appendix A. All these classifications say 
something about outcomes and about organizing 
outcomes, and provided useful^ input to .the 
current NCHEMS effort in this area, but several ^ 
problems are present. Many of the classifications 
found appeared to be quite arbitrary in their ; 
content and organization, and had their basis in 
other than empirical studies. Those that are 
empirically based tend to be narrow i^ their area 
of focus, for example, classifying only particular 
kinds of student outcomes. Furthermore, most of 
the broader classifications consist only of simple 
lists of categories, and those remaining tend to go 
into little if any definitional detail. None of the . 
broadly focused outcome classification systems 
cover the full range of potential postsecondary 
education outcomes, therefore, this new attempt 
was made to structure outcomes in a useful way, 
and .the proposed system is described in this 
chapter. 

In certain past attempts to structure educational 
outcomes, educators wiin particular philosophical 
and theoretical orientations have felt that the 
structure.favored other philosophies or theories of 
education and was biased against their own 
^because of the nature of its organisation and/or 
' content. Therefore, special care was taken to 
.eliminate value and philosophical connotations as 



^ For •xampl«, tarw could tm uMd at a dlmantton for •n^outcomM 8tn>ctum . 
and could h«vt catagortM of ''short tarm" and "k>n0 tarm" or of TIma 1 , Tima 2. 
Tima 3. and so forth. Othar potalbla dlmanalona hava baan outllnad by 
Goodman (1971) and Includa obaarvablllty, aconomic/nonaconomic, 
miiturablllty, quantltatlva/quaJltaltva, Indlvldual/aoclat raluma. and Intamal/ 
aittmal raluma. 



much as possible from the NCHEMS Structure. It 
Ts probably impossible to eliminate such bias 
completely, however. 

An in-depth discussion of the concept of 
''educational outcome" that underlies the 
structure presented here was giyen in Chapter 2. A 
number of the attributes and other factors 
discussed there, each contributing to an 
understanding of a particular educational 
outcome^ were consijdered for use as a dimension 
of the NCHEMS O'utcpmes Structure: form, 
change^status, focus, output/impact, audience, 
functional area, and time. Two of the factors were 
selected as appropriate dimensUms for the 
Structure: "audience", and "time./Two of the 
other factors discussed in Chapters, ''focus" and 
"change status," were combinecA into a third 
formal dimension for the structure, labeled 
"type-of-outcome." Figure 3.1 shows these three 
dimensions in relation to one another, and 
diagrams the NCHEMS Outcomes Structure in its 
simplest and broadest form. Figure 3.2, in turn, is 
a diagrammatic overview of the Structure that 
shows these dimensions^ with their major 
categories added. ^ Each of the dimensions is 
discussed separately in the following sections of 
this chapter, andfthen ,the entire Outcomes 
Structure is described4(i^more detail. 

Outcome category code numbers identitying 
particular outcomes are^.used throughout this 
chapter. As will be discussed later, the code 
numbers assigned to a particular outcon\e 
consist of three sets of numerical dIgHs separated 
by periods, where the first set identifies the 
.audience category, the second set the type-of- 
outcome category, .and the third set the time 
category into which the outcome falls, as shown 
below: . 



Audience 



Type-of-Outcome 
f Time 

xx.xxxx.xx. 



y Note In tha thraa-dlmanslonal matrix of Plgura 3.2 that calls vb formad 
which corraspond to a spaclfic "audlanca" catagory, a apaclfic 
'lypd-of^outcoma" catagory. and a "tima" catagory. Thraa axamplaa hava bean 
ahowri(indtcatadbyX't). 
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A coding scheme has several important uses in a 
structure such as this. First it provides a 
shorthand for referring to specific cells in the 
matrix formed by the three dimensions. For 
example, it is much more efficient and easy to 
write "20.2000.10 outcomjfe". or to say "Cod^ 
twenty-point-two thousand-point-ten outcoipes" 
than to write or say "knowledge, technolpgy, and 
art form outcomes occurring during Time 1 fbr 
interest-based ccrnmunities.*^ This is true 
especially if one is classifying a targe number of 
specific outcomes, outcome information items, or 
outcome measures at a fine level of detail. 
Second, st'ch numbers provide a readily available, 
consistent, and meaningful computer storage 
location code for every category of outcome in the 
Structure, for computerized, or manual, outcome 
information retrieval systems. 



The "^Audience" Dimension 

This dimension is intended to aid in the 
^ classification of outcomes by identifying who or 
what receives or is affected by the outcome of 
concern. That is, it is designed to determine the 
person, community, or other entity that actually 
receives or is affected by (or is intended to receive 
or be affected by) a particular output or impact. 
For the purpose of the current version of the 
Outcomes Structure, the categories and sub-, 
categories (second-level categories) listed in 
Figure 3.3, along with their associ^t^. code 
numbers, will be used to^ classify outcomes 
according to audience. 



V 



For many planning purposes, a third level of detail 
is often needed for some second-level audience 



Figure 3.1 

ILLUSTRATION OF THE THREE DIMENSIONS OF THE NOHEMS OUTCOMES STRUCTURE 



AUDIENCE 
DIMENSION 

(Who or what Is 
intended to, or actu- 
ally does, receive or 
be affected by the 
outcome?) 




TIME DIMENSION 

(When is the out- 
come expecied to 
occur, or when does 
it occur?) 



TYPE OF OUTCOME DIMENSION 

(What basic entity is, or is intended to 
be, maintained or changed?) 
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Figure 3.2 

DIAGRAMMATIC OVERVIEW OF THE NCHEMS OUTCOMES STRUCTURE 



Ah Ecoopmic Outcome 
occurring aX Time 2 for 
Interest-Based Communi- 
ties 




AUDIENCE 
DIMENSION 

(Who or what is 
intended to, or actu- 
ally does, receive or 
be affected by the 
outcome?) 



to 

CO 



A Knowledge. Technology, 
and Art Form Outcome 
occurring at /Time 1 for 
Individuals/Groups. 



Individual/ 
'Group Clients 



Interest-Based 
Communities 



- Geographic- 
Based 
Communities 



Economic Outcomes 

i 
I 
I 



i 1 

! Human Characteristics Outcomes 



I I 

I I 

I Knowledge. Technology, and 

Art Form Outcomes 
I I 

1 . I 



I 



Resource and Service 
Provision Outcomes 



I 
I 



Aggregates 
of People 



4. — 



t 
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TIME DIMENSION 

(When is the out- 
dome expected to 
occur, or when does 
It occur?) 



A Knowledge. Technology, 
and Art Fojjm Outcome 
occurrli^.^ Time 2 for 
Aggregates of People. 



TYPE OF OUTCOME DIMENSION 
(What basic entity is, or is intended to be, maintained or changed?) 



Figure 3,3 " 
THE CATEGORIES AND SUBCATEGORIES OF THE AUDIENCE DIMENSION 

10. IndMduif/Qroup C//en«— This.category refers to persons or oroups of persons who are direct clients of the postseoondary education unit of 
concern and/or their Immediate associates, such as family and relatives or peers. ^ 

^ Sfucteflfs— Individuals or groups of Individuals who currently are enrolled In the program, Institution, or system of posteecondary education. 

12. Formf SfurfMfs-lndlvlduals or groups of Individuals who formerly were enrolled In the program, Institution, or system of postsecondary 
' education. 

i 

1 3 . Family and fle/af/ves of Studwts or Former $tudef}ts 
° 14. P9ers and A ssociates of Students or Former Students 

15 Faculty * 

16. Staff Other than Faculty ^ 

17 Other Individual /Group Clients^kn example would t>e an Indlvldualw^io Is none of the above but Is served by an advisory service offered 
by the college. "s. ^, 

20. interest-Based Communities— Ihls category refers to large groups that are Identified as entitles worVIng toward a mil-defined interest or 

mission. 

, 21 . Private Enterprise Communities— CommuniUes where a major purpose Is financial remuneration and prof It— for example, corporations, 
small businesses, and farmers 

22 Association Commun/f/es— Communities where members belong on the basis of affiliation rather than employrnent. such as unions and 
professional societies. , < 

23. Govemmenf Cofnmun/f/es— Co» "munitles designed to administer government regulat{6ns and services, such 'as city nail, state department 
^ of education, and legislative communities. ^ ^ 

24. Nongovernmental/Public Sen/tee Communities Other than the lnstitution,Producmjf the Oyfcome— Nonprofit service organizations, such 
as schools, hospitals, welfare agencies, philanthropic foundations, colleges (other than the college producing the outcome), and research 
organl2atlons 

25. Institution or Institutional Unit Producing the Oufcome— The postsecondary education Institution and/or units within that Institution that 
are perceived as the producer/facilitator of the outcome(s) of concern. 

26 Other Interest-Based Communities— An example would be an ad hoc coalition task force of representatives from two or more of the above 
areas. . 

30 Geographic-Based Communities—ms calegory refers to lagje groups defiiied on the basis of functional territortel boundaries. ' 

31 Local Community—k township, city, county, metropolitan area or other type of locality having panicular boundaries. It Is not necessarily 
restricted lo the legal or|urisdlctionai boundary, but the functional one in-«hich the impact cf the institution is (or should be) directly and 
physically felt. The boundaries will vary with the institution/program and outcome of concern. 

32 The State 

33. A Region— An aggregation of states or paris of states. 

34. The Nation 

35. An international Community r. ^ 

I? ' 

36 Other Qeographic-Based Commumties—An example would be £ research discovery that affects primarily people living in the coldest 
latitudes, or where it snows heavily. 

40 Aggregates of People— IMs category refers to subpopulations of people distinguished by particular characteristics that may indicate common 
concerns, needs or wants, but ^ho do not necessarily have a common interest or mission, ano therefore do not constitute communities. 

* 41. • Ability Level Subpopy/af/ons— Subpopulations defined according to level of ability/ proficiency, on general intellectual functioning or 
• specific skills— for example, gifted, typical, disadvantaged, or skilled, semi-skilled, unskilled. 

42. A ge Subpopulations 

43. Educational Level Subpopulations 
44 Income Level Sub^pulations 

45. Occupation Subpopulaiions 

46. Physical Disability Condition Subpopulations 

47. Race Subpopulations 

46.- Sex Subpopulations , 
49. Other Such Aggregates 

50 Other 4 utf/ences- Examples would be the natural environment that is affected by university -sponsored research (which in turn would be 
expectod lo have impacts on audiences such as Individuals and communities) and popuialions of animals (such as the animals affected by 
efforts to keep depleted species from becoming extinct or by the developrrwnt of veterinary medicines). 
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categories. The third-level categories needed vary 
with the producer/facilitator levei of analy^s— for 
example, institutionwide administrators may need 
to examine student outcomes separately for 
different degree-aspiration groups, while program 
administrators may need to examine student 
outcomes separately for those majoring Jn the 
program and those only taking courses in the 
program. Similarly, they can vary according to 
functional concern— for example, at the program 
level one administrator; may need to compare 
full-time to part-time students, another may need 
to compare according to whether or not they are 
majoring in the program, and a third may need to 
compare according to whether or not the students 
are oisadvantaged socioeconomically or other- 
wise. See Appendix C for procedures to use in 
adding a third level of categories to this 
dimension that will meet the planner's needs. 



The "Typ0-Of-Outcom©" Dimeri^ion 

The type-pf-outcome dimension identifies Whether 
or not the outcome involves a change in status 
(maintenance yersus change), and the "focus" of 
the maintenance or change (the basic entity that 
is maintained or changed, such as knowledge and ' 
understanding^ skills and competencies, attitudes 
and values, certification status, income, standard 
of Hying, social interactions, and art forms or« 
expression). Four levels of specificity are 
included iti the Structure for this dimension, 
which means that any category of "type-of- 
outcome" can be identified by a four-digit code 
number. The five major categories (^e top-levei 
categories) are coded and defined as follows: 



Category 
Code Number 


The Major *Type-of-Outfcome" 
Category Names and Definitions 


1000 


Economic Oufcomes— Maintenance or change in economic characteristics and 
conditions of individuals, groups, organizations, and communities,' e.g., in 
economic access, in economic mobility and independence, in economic security, 
and in incbmeand standard of living. 


2000 


Human Characteristic Of/fco/77es--;Maintenance or change in human makeup and 
oiiaiacierioiics \oin6r man Knowiouye aiia unQersianQtng; ot mQiviQuais, groups, 
organizations, and communities, e.g., aspirations, competence and skills, affective 
characteristics, perceptual characteristics, physical and physioiogicai character- 
istics, personality and personal coping characteristics, recognition and certifica- 
tion, and social roles. 


3000 


Knowledge, Technology, and Art Form Outcomes— Maintenance or change in the 
knowledge and understanding, technology, or the art forms and works possessed 
or mastered by individuals, groups, organizations, and communities, e.g., 
discoveries and inventions, technical developments, syntheses'and reformulations 
of knowledge, . new schools of thought in art and works created in those new 
traditions, renovation of art works. 


4000 


Resourcejand Sen/ice Provision Outcomes— Maintenance or change in the direct 
resources and services (other than those included. above) provided to individuals, 
, groups, organizations, and communities, e.g., "providing facilities, events, 
advisory assistance, analytic assistance, teaching, health care, and leadership. 


5000 


Other Maintenance and Change Oufcomes—Examples would be: maintenance or 
change in the format, arrangement, activity, or administrative operation of an 
organization or institution; maintenance or change in the aesthetic/culturaf level 
of the local community; maintenance or change in family or community activities, 
practices, and traditions. 
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Xigure 3.4 lists the names and code numbers for 
all second- and third-level categories of the "type" 
dimension, separately for each top-level category. 
Definitions for each of these more- detailed 
categories, along with examples of indicators 
or measures for each, specific type of outcome, 
are given in Appendix B. The majority of the 
outcome indicator examples listed in Appendix B 
concern outcomes for individuals, but it should be 
remembered that composites of these are in many 
cases outcomes for communities and other 
populations, for example, geographic mobility, 
economic ^security, and annual or lifetime 
earnings. 

Th^ fourth level of detail for this dimension 
(identified by the fourth digit in the type-of- 
outcome code number) consists of two categories 
that have already been referred to throughout the 
discussion of the dimension, namely, "mainte- 
nance" and "change." These two types of outcome 
goals require quite different orientations and 
Approaches in planning and management, and 
they have been considered by different people 
throughout the decades to represent the 
fundamental purposes of postsecondary education 
(for example, preserving the culture versus 
improving the condition of mankind). Too often 
planners think only of bringing about change, and 
do not even think to consider maintenance 
outcomes in their planning. Similarly, many times 
educators have failed to distinguish whether a 
particular goal is focusing on maintenance, 
change, or both maintenance and change^this in 
spite of the quite different approaches and 
orientations required when emphasizing mainte- 
nance as compared to emphasizing change. For 
these reasons, it was decided to make these 
categories a part of this dimension. Definitions 
'for each category follow: 



xxxl. Maintenance— Outcomes that result in 
keeping the status quo; in stabilization, 
reproduction, or preservation. Examples 
include preserving cultural values, restora- 
tion of , artifacts and paintings, keeping up 
the educational level of the family, and skill 
maintenance provided by in-service educa- 
tion. 

xxx2. Change— Outcomes that result in alteration 
of the status quo; in reorganization, 
modification, revision (improvement or 
otherwise), or replacement. Included are 
economic and social mobility, degree or 



certification awarded the student, increased 
knowledge and skill level, new art forms, 
technological innovations, medical dis- 
coveries, and so forth. 

As phown above, a "type dimension" fourth-level 
digit of "1" indicates a maintenance outcome, 
while a fourth-level digit of "2" indicates a change 
outcome. As is true for the categories at any other 
level of detail, a zero indicates "no distinction for 
the categories at this level" or "both the 
categories at this level apply." 

Some people, while acknowledging the historical 
social importance for higher education of the 
"maintenance" concept of an educational 
outcome, have questioned its practical, direct 
usefulness in administrative planning for 
institutions and programs. Therefore, perhaps an 
example should be cited. Lenning, Munday, and 
Maxey (1969) conducted a study where they 
examined student educational growth, adjusted 
for initial leveL of ability, using analysis of 
covariance, in four subject areas: English, 
mathematics, social studies, and natural 
sciences. Student educational growth was 
operationally defined as change in ACT retest 
score for each area after two years of college 
experience. For each of five colleges and 
universities, the educational growth in each 
subject area was compared for those taking 
courses in the area to those not taking courses in 
the area. It was discovered at one private liberal 
arts college that both groups decreased 
appreciably on retest in the area of mathematics-* ^ 
exhibited negative educational growth. In' 
conversing with institutional officials, it was 
discovered that many of their students taking 
math went directly into calculus. Since the test 
measures primarily basic mathematics skills, such 
as algebraic manipulation, a reasonable hypothe- 
sis for the negative growth in math seems 
apparent: their mathematics students probably 
were getting little or no practice in using basic 
math skills in the more advanced math courses 
they were taking, and they were becoming "rusty" 
on those skills as a result. Such findings on a 
campus may suggest to mathematics faculty that 
they should consider buiiding some periodic, short 
practice exercises using basic math skills into the 
more advanced mathematics courses, so that the 
students maintain proficiency in those skills. 

The categories and subcategories for this "type- 
of-outcome" dimension are based on and extend 
from a variety of previous work. Much careful 
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Figure 3.4 

CODED LISTING OF THE SECOND- AND THIRD-LEVEL SUBCATEGORIES 
FOR EACH FIRST-LEVEL CATEGORY OF THE TYPE-OF-OUTCOME DIMENSION 



Cdteoory 

Code Number Entity Being Maintained or Changed 


Category ^^-* 
Coda Number Being MamTalned or Changed 


1000 ECONOMIC OUTCOMES 

1100 Economic Access and Independence Outcomes 
1110 Economic Access 

1120 Economic Flexibility, Adaptability, and Security 
1130 Income and Standard of Living 

1 200 Economic Resources and Costs 

1210 Economic Costs and Efficiency 

1220 Economic Resources (including employees) 

1300 Economic Production 

1310 Economlc'Productlvity and Production 
1320 Economic Services Provided 

0 

1400 Other Economic Outcomes 


2000 HUMAN CHARACTERISTIC OUTCOMES (continued) 
2780 Power and /or Authority 
2770 Job, School, or Life Success 

2780 Other Status. Recognition, and Certlti«:atlon Outcomes 
2800 Social Activities and Roles 

2610 Adjustment to Retirement 
2820 Affiliations 

2830 Avocational and Social Activities and Roles 
2840 Career and Vocational Activities and Roles 
2850 Citizenship Activities and Roles 
2860 Family Activities and Roles 
2870 Friendships and Relationships 
' 2880 Other Activity and Role Outcomes 

2900 Other Human Characteristic Outcomes 


2000 HUMAN CHARACTERISTIC OUTCOMES 

2100 Aspirations 

' 21 1 0 Desires, Aims, and Goals 
2120 Dislikes, Likes, and Interests 
2130 Motivation or Drive Level 
2140 Other Aspirational Outcomes 

2200 Competence and Skills 
2210 Academic Skills 

2220 Citizenship and Family Membership Skills 
2230 Creativity Skills \ 
2240 Expression and Communication Skills 
2250 Intellectual Siyils 

2260 Interpersonal, Leadership, and Organizational Skills 
2270 Occupational and Employability Skills 
2280 Physical and Motor Skills 
2290 Other Skill Outcomes 

" 2300 Morale, Satisfaction, and Affective Characteristics 
2310 Attitudes and Values 

2320 Beliefs, Commitments, and Philosophy of Life 
2330 Feelings and Emotions 
2340 Mores, Customs, and Standards of Conduct 
2350 Other Affective Outcomes 

2400 Perceptual Characteristics 

2410 Perceptual Awareness and Sensitivity 
2420 Perception of Self 
2430 Perception of Others 
2440 Perception of Things 
2450 Other Perceptual Outcomes 

2500 Personality and Personal Coping Characteristics 
2510 Adventurousnessand initiative 
2520 Autonomy and Independence 
2530 Dependability and Responsibility 
2540 Dogmatic/Open-Mlndeo, Authoritarian/Democratic 
2550 Flexibility and Adaptability 
2560 Habits 

2570 Psychological Functioning 
2560 Tolerance and Persistence 
2590 Other Personality and Personal Coping Outcomes 

2600 Physical and Physiological Characteristics 
2610 Physical Fitness and Traits 
2620 Physiological Health 
2630 Other Physical or Physiological Outcomes 

2700 Status. Recognition, and Certification 

2710 Completion or Achievement Award 
2720 Credit Recognition 
2730 Image, Reputation, or Status 
2740 Licensing and Certification 

2750 Obtaining a Job or Admission to a Follow-up Program 


3000 KNOWLEDGE, TECHNOLOGY, AND ART FORM OUTCOMES 

3100 General Knowledge and Understanding 

31 1 0 Knowledge and Understanding of General Facts and 
Terminology 

- 3120 Knowledge and Understanding of General Processes 
3130 Knowledge and Understanding of General Theory 
3140 Other General Knowledge and Understanding 
3200 Specialized Knowledge and Understanding 

3210 Knowledge and Understanding of Specialized Facts 

and Terminology 
3220 Knowledge and Understanding of Specialized 
Processes 

3230 Knowledge and Understanding of Speclsiized Theory 
3240 Other Specialized Knowledge and Understanding 

3300 Research and Scholarship 

331 0 Research and Scholarship Knowledge and 

Understanding 
3320 Research and Scholarship Products 

3400 Art Forms and Works 
3410 Architecture 
3420 Dance 
3430 Debate and Oratory 
3440 Drama 

3450 Literature and Writing 
3460 Music 

3470 Painting, Drawing, and Photography 
3480 Sculpture 
3490 Other Fine Arts 

3500 Other Knowledga, Technology, and Art Form Outcomes 


4000 RESOURCE AND SERVICE PROVISION OUTCOMES 

41C0 Provision of Facilitiesand Events 
4110 Provision of Facintles 
41 20 Provision or Sponsorship of Events 

4200 Provision of Direct Services 
4210 Teaching 

4220 Advisory and Analytic Assistance 
4230 Treatment, Care, and Referral Services 
4240 Provision of Other Services 

4300 Other Resource and Service Provision Outcomes 


5000 OTHER MAINTENANCE AND CHANGE OUTCOMES 
5100 Aesthetic-Cultural Activities, Traditions, and Conditions 
5200 Organizational Format. Activity, and Operation 
5300 Other Maintenance and Change 1 



The foudh-iev«lcatooor<es. into which any ottr>e categories I'Sted here can twdrvided . «re 'm«mten ta fourth dtQtt of r)aod change (a fourth digit of "2") 
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work went into their formuiation .^However, some 
users of the Structure may wish to bypass 
particuiar ieveis, or combine or modify some of 
the categories at those ieveis, dnd the Structure is 
designed to faciiitate such modification. It is 
important that a structure such as this be fiexibie 
andmodifiabie. 

The "Time" Dimension 

Time has aiways been considered an especiaiiy 
important factor for many student outcomes. 
Particuiar teaching content and activities do not 
have much effect untii the time a student is 
"ready" for them, and there are presumabiy 
specific times when curricuium materiais and 
emphases of different types wiii have maximum 
infiuence. For exampie, an understanding of the ^' 
principies of advanced caicui.us cannot readiiy 
occur untii the stacjent has mastered basic 
freshman mathematics. Simiiariy, many. students 
may not be ready to choose a coiiege major untii 
after they have chosen a vocation. Another time 
consideration is that some educationai outcomes 
wouid be expected to occur eariier than others, 
and many wiii differ in how iong they iast or 
persist (for exampie, short-duratiorr outcomes 
versus iong-duration outcomes). Such factors 
have important impiications for goai setting and 
determining priorities, as weii as for other 
pianning activities. To iiiustrate, outcomes that 
occur at or prior to graduation are more often a 
direct focus of pianning than are impacts of those 
outcomes aftergraduation, because the institution 
has some direct controi over them. On the other 
hand, if the reiationship of those outcomes to the 
later impacts can be ascertained, the iater impacts 
may aiso have some infiuence on priorities and on 
communication to the pubiic about outcomes. As 
stated by Havighurst (1952), to support his 
time-reiated concept of developmental tasks: 

There are two reasons why the concept of 
developmental tasks is useful to educators. 

3 The "focus" categories and subcategories (the top three levels of categories 
for the "type-of-outcome" dimension) developrr^ent started from the NCHEMS 
Inventory of Higher Education Outcome Variables and Measures (In MIcek and 
WaHhaus, 1973) and Involved a synthesis and modification of many "type" 
classifications found In the Nterature. with particular Influence being exerted 
by classifications based on or tested out using the results of educational, 
psychologlca). and sociological research Astin (1970). Bloom (1956). 
Brubacher (1968), Chlckering (1969). Dressei and Mayhew (1954). French 
(1957). Goodman (I37l). Gronlund (l97l). Gross and Grambsch (1968). 
Havighurst (1952). Healy et al (1971). Holland and Richards (1965). Klingberg 
(1970). Krathwohl et aL (1964). Lenning et al. (1974. 1975). Pace (1972). 
Peterson (1971). Schalock(l972). Schalock et al. (1972). and Vernon (i960). In 
deciding among discrepancies, and when definitions of terms were needed 
that dictionaries could not provide, text books in the pertinent discipline were 
referred to. and technical specialists were consulted as needed The "change 
In status" categories (the fourth-level categories) for this dimension were 
primarily motivated by formulations of Derr (1973) and Parsons (1951). 



First, it helps in discovering and stating the 
purposes of education in the schools. 
Education may be conceived as the effort of 
the society, through the school, to help the 
individual achieve certain of his develop- 
mental tasks. The secohd use of the concept 
is in the timing of educational efforts. When 
the body is ripe, and society requires, the self 
is ready to achieve a certain task, the 
teachable moment has come. (p. 5] 

It is not surprising, then, that ^Schwartz and 
Tiedeman (1957) included a time dimension in 
their general classification of educational 
outcomes, as did Astin (1970) in his taxonomy of 
student output measures in higher education. 
Although requiring different time categories from 
those relevant for student outcomes, time is 
important also for planning and analysis of 
nonstudent outcomes— for example, research and 
scholarship outcomes, art forms and works 
outcomes, and economic outcomes for society. 
Thus, "time" was made a dimension of the 
NCHEMS Outcomes Structure. Agreement could 
not be reached on time categories to include in 
the dimension, however, so it has no defined 
categories (just Time 1, Time 2, Time 3, and so 
on). 

For student outcomes, strong proponents can 
probably be found for any one of the following 
four categorizations: (1) year by year; (2) Astin's 
(1970) suggestion for use in analysis of "end of 
sophomore year," "at graduation," "one year out 
of college," and succeeding points in time; 
(3) Havighurst's (1952) categories of "adoles- 
cence," "early adulthood," "middle age," and 
"later maturity"; ^and (4) "short-term outcomes" 
and "long-term outcomes," where the defmed 
boundary between the two classes depends on the 
situation and perceptions of the claasifier. 
Undoubtedly some people would favor categoriza- 
tions other than the possibilities mentioned 
above, including combinations and modifications 
of those cited. 

Another problem, even it the categories are 
limited to student outcomes, is that the time 
categories of interest are different depending on 
the philosophy of the user and the particular use 
or need in his or her context. To illustrate, a 

^ The potential importance of long-term follow-up of college graduates (lor 
example, at "middlo age") is great. For example, in a personal conversstion 
with the senior author of this document. David R. Witmer reported that his 
current research has found income at age 40 to be the best predictor of social 
economic rate of return from college attendance. 
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curriculum developer will probably need a quite 
different set of time categories than will a 
campus administrator concerned with general 
campus-wide outcomes. For these reasons, it was 
decided to leave any specific categorizing (and 
definitions for those categories) up to each 
individual using the Outcome^ Structure. For 
some purposes, users of the Structure may decide 
not to use the time dimension of th^ Structure, or 
they may decide to use it for only certain types of 
outcomes {for example, for student impacts, but 
not for student outputs or for nonstudent 
outcomes). 

The NCHEMS Outcomes Structure 

This section includes a more detailed overview of 
the NCHEMS Outcomes Structure than was 
shown in Figure 3.2 on page 23. Since the "time" 
dimension was not assigned defined categories, 
the focus of discussion here will be on the matrix 
formed by the intersection of tha>"audience" and 
"type-of-outcome** dimensions. This does not 
negate the importance of the "time" dimension, 
however, nor the fact that at least some users of 
the Structure will want to decide on appropriate 
time categories and classify outcomes along that 
dimension also. As implied in the preceding 
section, time is 9S essential a dimension from 
which to view educational outcomes as either of 
the other two dimensions of the Structure. 



Figure 3.5 is a more detailed drawing of the 
audience/ type-of-outcome matrix than shown in 
Figure 3.2. Note that there are a total of 696 cells 
even though the third- and fourth-level categories 
for the "type-of-outcome" dimension are not 
represented in the drawing. Such a large number 
of cells is perhaps too many for anyone to keep 
track of, iet alone the 5,220 subcells resulting 
when the additional two levels of detail for the 
"type-of-outcome** dimension are added. But it 
should be remembered that the majority of those 
using the Structure will probably be concerned 
with only one or two of the major cells, and 
perhaps only some of the subcells within each of 
those cells will be of concern. By glancing 
through the audience and type-of-outcome 
category labels, the user of the Structure can 
quickly discover the cells and subcells on which 
attention should be focused. 

As demonstrated in Figure 3.5, putting the code 
numbers for the two dimensions together 



(separated by a period), with the "audience** ,code 
listed first, gives codes for the cells. Reference to 
Figure 3.4 or Appendix B allows even more detail 
than Figure 3.5. Examples of more detailed 
outcomes than shown in Figure 3.5 are code 
number 11.2282 (change in physical and motor 
skills, for students) and code number 31.2241 
(maintenance of expression and communication 
skills in the local community). 

For those wishing to also classify according to 
"time,** a'^third set of figures could be added. 
Suppose, for example, that a user of the Structure 
developed the following categories and subcate- 
gories for "time,** and assigned the indicated code 
numbers to each, which could be included wjth 
the code numbers for the other two dimensions 
(as a third set of numbers) to classify outcomes. 

0 



10. Short-Duration Outcomes 


11. Short-duration 


12. Short-duration 


outcomes appear- 


outcornes appear- 


ing at or prior to 


ing after gradu- 


graduation 


ation 



20. Long-Duration Outcomes 


21. Long-duration 


22. Long-duration 


outcomes appear- 


outcomes appear- 


ing at or prior 


ing after gradu- 


to graduation 


ation 



Examples of code numbers in this three-dimen- 
sion classification follow: 13.1002.20— long- 
duration economic change outcomes for family 
and relatives of students or former students, 
13.1002.22— long-duration economic change out- 
comes appearing after graduation for family and 
relatives of '^tudents or former students, 
13.1132.22— long-duration change in income and 
standard of living appearing after graduation for 
family and relatives of students, or former 
students, and 31.3001.21— long-duration knowl- 
edge, technology, and art form maintenance 
outcomes appearing at or prior to graduation for 
the local community (an example would be art 
students at the college helping local community 
residents preserve art objects). 
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Figure 3.5 

A'MORE DETAILED DRAWING OFTHE AUDIENCE/TYPE-OF-OUTCOME MATRIX 



Code Number 11.2700 = Status. Recognition, 
and Cerlincetlon Outcomes lor Students. 
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For some purposes of postsecondary aducation 
decision making, the amount of detail shown in 
Figure 3.5 will be sufficient. For other purposes, 
however (and especially within the institution), all 
four levels of detail within the "type" dimension 
are needed— for example, in communication 
about very specific outcomes, in computer 
storage and retrieval of specific outcome 
information, in curriculum development, in 
planning for a course that focuses on just a few of 
the cells in the Outcomes Structure, and in 
developing a comprehensive list of specific 
outcome-oriented objectives in a broad area. 

For some of these purposes, such as certain 
curriculum development and formulating outcome 
objectives for a course, even more detail may be 
needed than provided by the finest categories in 
the "type-of-outcome" dimension of the Structure. 
Suppose, for example, that a major objective in 
one program is to change student attitudes 
(Outcome Category 11.2312). The person may 
wish to subdivfde this Into the still finer 
categories of political attitudes, racial^ethnic 
attitudes^ religious attitudes, personal ethics, and 
so forth, before trying to generate lists of specific 
outcomes. Other users of the Stru ;ture may need 
more detailed subcategories the audience 
dimension (as discussed earlier), or for both 
dimensions. Appendix 0 discusses and gives 
guidelines for adding subcategories to any 
dimension of the Outcomes Structure (extending 
the structure). That appendix is meant only for 
those vjho determine that the Structure Is not 
detailed enough in certain areas for their 
purposes. 



Summary 

A classification system for postsecondary 
education outcomes, called the Outcomes 
Structure, was developed that has three 
dimensions labeled as follows- "Audience," 
"Type-of-Outcome," and "Time." The first two 
dimensions have been assigned various categories 
and subcategories.' No set categories that would 
be acceptable to most people in most 
postsecondary education contexts could be 
ascertained for the third dimension, "Time." 

The "Audience" dimension of the Outcomes 
Structure refers to the things or persons intended 
to, or that actually do, receive or be affected by 
the outcome of concern. The categorie.s and 



subcategories assigned to the "Audience" dimen- 
sion are as follows: Individual/Group Clients 
(students, former students, family and relatives of 
students or former students, peers and associates 
of students or former students, faculty, staff other 
than faculty, other individual/group clients), 
Interest-Based Communities (private enterprise 
communities, association communities, govern- 
mental communities, nongovernmental/public 
service communities other than the institution 
producing the outcome, institution or institutional 
unit causing the outcome, other interest-based 
communities), Geograptiic-Based Communities 
(local community, the state, a region, the nation, 
an international community, other geographic- 
based, communities). Aggregates of People 
(subpopulations based on ability levpl, age, 
educational level, income level, occupation, 
physical disability condition, race, sex, and other 
such aggregates), and Ottter Audiences. 

The "Type-of-Outcome" dimension identifies the 
basic entity the outcome is focusing on, and 
whether or not the outcome of concern involves a 
change in status. The five major categories of 
"Type-of-Outcome" are listed below alorig with 
their associated subcategories. In addition, each 
of these categories and subcategories can be 
Divided into "maintenance" and "change." 

Economic Oufcomes— Economic Access and 
Independence Outcomes (access; flexibility, 
adaptability, and security; Income and 
standard of living). Resources and Costs 
(costs and efficiency; resources). Production 
(productivity and production; economic 
services provided), and Other Economic 
Outcomes. 

Human Ctjaracteristic Oufcomes— Aspira- 
tions (likes and dislikes; interests; desires, 
aims, or goals'; motivation or drive level; 
other aspirational outcomes). Competence 
' and Skills (academic; citizenship and family 
membership; creativity; expression and 
communication; intellectual; interpersonal 
and leadership; occupational; physical and 
motor; other skill outcomes). Morale, 
Satisfaction, and Affective Characteristics 
(feelings and emotions; mores, customs and 
standards of conduct; attitudes and values; 
beliefs, commitments and philosophy of life; 
other affective outcomes). Perceptual Char- 
acteristics (perceptual awareness and sensi- 
tivity; perception of self; perception of 
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others; perception of things; other perceptual 
outcomes), Personality and Personal Coping 
Characteristics {adventurousness and initia- 
tive; autonomy and independence, depend- 
ability and responsibility; dogmatic/operf- 
minded, authoritarian/democratic; flexibility 
and adaptability; habits; psychological 
functioning; tolerance and persistence; other 
personality and personal coping character- 
istics outcomes). Physical and Physiological 
Characteristics (physical fitness and traits; 
physiological health; other physical or 
physiological outcomes). Status, Recogni- 
tion, and Certification (admission to a 
follow-up program or job; completion or 
achievement awards; credit; image, reputa- 
tion, or status; licensing; power and/or 
authority; other status, recognition and 
certification outcomes), and Social Activities 
and Roles (adjustment to retirement; 
affiliations; avocational and social activities 
and roles; career and vocational activities and 
roles; citizenship activities and roles; family 
activities and roles; friendships and 
relationships; ^'other activity and role 
outcomes). 

Knowledge, Technology, and Art Form 
Oufcomes—G'eneraL Knowledge and Under- 
standing (general facts and terminology; 
general processes, general theory). Special- 
ized Knowledge and Understanding (special- 
ized facts and terminology; specialized 
processes; specialized li^eory). Research and 
Scholarship (research and scholarship 
knowledge and understanding; research and 
scholarship products), and Art Forms and 
Works Outcomes (architecture; dance; 
debate and oratory; drama and acting; 
literature and writing; music; painting, 
drawing and photography; sculpture; other 
fine arts). 

Resource and Service Provision Outcomes- 
Provision of Facilities and Events (provision 
of facilities; provision or sponsorship of 
events), Provision of Direct Services 
(teaching; advising, analytic assistance; 
treatment, care, and referral services; 
provision of other services), and Other 
Resource and Service Provision Outcomes. 



and Operation; Other Maintenance and ' 
Change. 

Most users of the Outcomes Structure will need to 
focus on on*y a few categories of the structure. To 
determine the specific categories of concern, it is 
best to .start with the braodest categories and 
narrow down from them. For those who need to 
separate outcomes into classes through use of a 
•coding scheme, a code number is provided for 
every category and subcategory in the Structure. 
Furthermore, provision is made for the user of the 
Structure to subcategorize even further for one or 
more of the most specific subcategories within 
any category, and to assign code numbers to 
those added subcategories. This process has 
been called "extending the structure," and 
involves the use of other taxonomies and/or 
logical analysis. 

Because the Outcomes Structure is intended to be 
flexible and adaptable, it is appropriate to modify 
it by eliminating unimportant (to the user) 
categories and subcategories, modifying particu- 
lar categories and subcategories, substituting 
new dimensions for any of the three proposed, 
eliminating a dimension, or leaving ojt the code 
numbers. It should be recognized, however, thcit 
modifications may preclude using information 
derived from the Structure for comparisons 
outside the inslitution or agency making the 
modifications. 



Other Maintenance and Change Outcomes— 
Aesthetic-Cultural Activities, Traditions .and 
Conditions; Organizational Format, Activity, 
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Chapter 4 
PRINCIPLES AND CRITERIA 
FOR DEVELOPING AND EVALUATING 
THE OUTCOMES STRUCTURE 



Introduction , 

A§ should be clear by now, the Outcomes 
N^tructure is an evolutionary ^yoduct. It rests upon 
significant lx}dies of previous work and, the 
authors hope, will continue to grow and develop 
as time passes. Furthermore, as is evident from 
Lennif\g'.<* review of previous attempts to structure 
outcomes (1977b), there are many dimensions 
that are\ logically defensible and that have 
potential \jsefulness as the frameworic for a 
classification of postsecondary education out- 
comes. \^ , 
\* 

The Guiding Principles and Criteria 

Because there were a l^rge number of potentially 
useful dimensions to choose from in developing 
the Outcomes Structure^ and because such a 
classification can be organized in a variety of 
ways,^it seemed necessary to formulate some 
overall guiding principles br criteria for the 
development of the NCHEMS Outcomes 

Structyre^ Several developmental principles for 

the Structure^^were^suggested intuitively by the 
needs of postsecondary education planners and 
managers: the principles of (1) practical utility 
and (2) congruency with other decision-making 
tools. Then, after reviewing literature In the field 
of taxonomy and studying the previous attempts 
to classify outcomes, seven other **guiding 
principles of classification" were added for use in 
developing the Structure: (3) comprehensiveness, 
(4) no overlap, (5) reliability, (6) neutrality, (7) hier- 
archy, (8) reality, and (9) flexibility. These various 
principles or criteria are discussed below. 

1. Practical Utility. .Chapter 1 outlined a number 
of important needs of planners and other 
decision makers in postsecondary education 
that potentially could be met by an outcomes 
structure. To recap, these include: (a) the need 
for a tool to organize, file, and retrieve 
significant volumes of outcomes information 



in a consistent and systematic way; (b) the 
need for agreed.-upon outcome terms and 
definitions for more effective communication 
of outcomes information to mapy diverse 
groups; (c) the need for a device to assist in the 
identification of outcomes for which adequate - 
quantitative measures are laclcing; (d) the need 
for aids to help postsecondary institutions 
improve their goal definition and the 
translation^ of their goals into concrete 
objectives; (e) the need for planners, and 
managers to show more succinctly how their 
institution differs from its counterparts; (f) the 
need for an aid in developing needs-assess- 
ment survey questionnaires aimed at different 
groups; and (g) the need for improved analysis 
and evaluation of institutional and program 
outcomes. The first principle for developing 
the Outcomes Structure was that it be 
designed to help meet these needs to the 
greatest extent possible. If an outcomes 
structure is to be a viable instrument, it must 
have reai and perceived practical utility, 
especially for decision maimers in post- 
secondary education. 

2. Congruency with Other Decision-Making 
Too/s. Related to the first principle of 
demonstrated practical utility is a second 
concerning the need for congruency with other 
commonly used decision -malting tools, 
including those that have been developed by 
NCHEMS and by others. In order to have 
maximum utility, the Structure should 
interface as much as possible and complement 
such tools or aids. Outcomes information has 
limited utility in and of itself and must be 
combined with other type's of infomnation for 
adequate understanding of various program- 
matic and institutional activities and their 
results. 

Because a large proportion of users of other 
NCHEMS products would be expected to use 
the NCHEMS Outcomes Structure, it was- 
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considered especially Important that the 
Structure Interface appropriately with planning 
and management tools developed by NCHEMS. 
(It should, as much as possible, also 
complement pertinent planning and manage- 
ment tools developed outside of NCHEMS, 
such as Pace's Kit, the ETS Institutional Goals 
Inventory, and the Battel needs assessment 
model.) The Program Classification Structure 
(Collier, 1978), which helps institutions and 
the postsecondary education systerfTs'of 
Institutions enumerate and organize their 
programs and activities, was discussed in 
Chapter 2. Another general framework 
designed to help organize the large bodies of 
information necessary for effective planning is 
the NCHEMS Program MeasOres (Topping 
and Miyataki, 1973). The Program treasures 
document was intended to aid users of the 
Program Classification Structure (PCS) by 
describing six categories of quantitative 
indicators that could provide information about 
each of the program elements lodged in the 
PCS: . 

1. Resource Measures— the physical and 
human resources utilized within the 

-program element during a stated time 
period. 

2. Financial Measures*— the source of funds 
and expenditures for physical and human 



resources utilized at a specified level of 
activity within a program element during a 
stated time period. 

3. Activity Measures--the level and type of 
operations carried on within a program 
element during a stated time period. 

4. Target Group Measures— the people, 
places, or things toward whom or at which 
the activities of a program, element are 
aimed or directed during a stated time 
period. 

5. Beneficiary Group Measures— the people or 
groups of peopfe who benefit directly or 
indirectly from either the activities or 
outcomes of a program element during a 
stated time period. 

6. Outcome Measures— the outcomes achieved 
or the products generated by the activitks 
of a program element during a stated time^ 
peiiod. 

It was intended that these various categories of 
program measures could be arrayed against 
the PCS categories and that each type of 
information would then be generated about 
each conrvponent of interest within the PCS. 
Thic two-way matrix is summarized in Figure 
4.1. 



Figure 4.1 
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While the above two NCHEMS tools focus on 
formal educational activities, a third tool 
focuses on the institutional environment. The 
NCHEMS ''Inventory of Institutional Environ- 
ment Variables and Measures" (Micek and 
Arney, 1974a) should have an especially close 
relationship with the Outcomes Structure. 
Environmental phenomena are closely inter- 
twined with outcomes for at least two reasons. 
One is the fact that the nature and degree of 
various outcomes is influenced and 'mitigated 
by these environmental phenomena. Thus, a 
student's satisfaction with his institutional 
program is conditioned in part by the 
surrounding environment of the institution. 

A second reason for considering environmental 
factors in close juxtaposition with outcomes is 
the fact that one person's environment may 
well be another person's outcome. This is true 
particularly in the support program area. One 
point of view would consider physical 
facilities; such as libraries or student housing, 
as a part of the environment that is a necessary 
support' for the instructional and research 
programs of the institution. On the other hand, 
institutions do allocate resources to the 
creation of this environment and thus it is, at 
least in part, an end or outcome in and of itself. 
One should note that our inadequate ability to 
measure outcomes directly quite often leads to 
♦he use of environmental phenomena as 
proxies for various outcomes. Thus, the 
number of books in a library's collection might 
wel! be considered a proxy for student growth 
and development outcomes. 

For the present, the Outcomes Structure and 
the Inventory of Institutional Environment 
Variables and Measures are treated as separate 
entities. This does not preclude the possibility 
that these two products will be combined into 
a single structure at some later stage of 
development. 

As noted in Chapter 1, the NCHEMS Inventory 
of Higher Education Outcome Variables and 
Measures (in Micek and Wallhaus, 1973) is a 
significant conceptual forerunner of the 
Outcomes Structure proposed in this docu- 
ment. The Inventory was a first attempt to 
develop a relatively complete characterization 
of educational institution outcomes and a set 
of operational indicators or measures 
associated with those outcomes. Structurally, 



the Inventory has the following major 
categories: 

Section 1.0: Student Growth and Development 
* Outcome Variables 

1 .1.0: Knowledge and Skills Development 
1.2.0: Social Development 
1.3.0: Personal Development 
1.4.0: Career Development 

Section 2.0: Development of New Knowledge 
and Art Form Outcome Variables 

Section 3.0: Community Development and 
Service Outcome Variables 

3.1.0: Comnmjnity Development 

3.2.0: Community Service 

3.3.0: Longer Term Community Effects 

As can l>e seen from the above, the I nventory Is 
somewhat traditional in that it accepts the 
major divisions of outcomes or activities 
typically associated with higher education 
institutions. This naturally imposes certain 
limitations that the present structure Is 
intended to remove. However, there' is no 
implication that the bulk of the outcomes 
covered by the Inventory are in any way 
unimportant, nor that the Inventory will no 
longer be useful to planners and managers. 
Indeed, one constraint imposed upon the 
development of the proposed Outcomes 
Structure was that it should permit the 
incorporation of all the outcomes identified in 
the Inventory of Higher Education Outcome 
Variables and ^\4easures. The Structure's 
*'type-of-outcome" categories con-esponding 
to each category of the NCHEMS Inventory are 
Jisted in Appendix 0. 

The Outcome Measures and Procedures 
Manual (Micek. Service, and Lee, 1975) is 
fundamentally a set of data acquisition 
procedures that are associated with only a 
subset of the full range of educational 
outcomes. The Outcomes Structure will serve a 
purpose in relation to this manual in that it will 
indicate which educational outcomes are 
susceptible to quantitative assessment and 
which are not. 

An additional NCHEMS product for which the 
Outcomes Structure has potential implications 
for its continued development is the Student 
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Outcomes Questionnaires (Bower and Ren- 
kiewicz, 1977 a and b). these validated sets of 
. data questionnaires and an associated manual 
are for use in different types of postsecondary 
institutions and ar6 designed to assist 
" institutional officials in conducting six* kinds 
of student outcomes studies— student entry 
. studies, 3tudents*in*progress studies, attrition 
* studies, program completer studies, short- 
term follow-up studies, and long-term 
follow-up studies. 

J A number of other NCHEMS planning and 
management tools potentially relate to t^e 
Outcomes Structure, but those discussed are 
•the ones of greatest significance. There are 
others, however, where future versions could 
be very much affected by what comes out of 
the Outcomes Structure. For example, the 
Information Exchange Procedures (to aid 
institutions in exchanging basic institutional 
i^nformation for, planning purposes) and the 
Academic Unit Planning and Management 
manual (to aid department heads in their 
planning and decision making) b6th have 
outcomes modules that were Influenced by 
previous NCHEMS outcomes*, products. Simi- 
larly, several tools being developed for 
state-level planners will have outcomes 
components, and outcomes will undoubtedly 
be a -concern in a number of planning and 
management tools developed by NCHEMS in 
the future. 

3. Comprehensive/^ess. One of the major 
problems with previous attempts at structuring 
outcomes was that they did not cover the entire 
array of possible educational outcomes. 
Therefore, a third principle for development of 
the Outconfies Structure was that it be as 
comprehensive as possible, or, as Wilhoyte 
.(1965) labels it, the principle of ''complete- 
ness." Derr (1973) used the term "exhaustive" 
and stated that two conditions were implied by 
this criterion: (if there should be a definite 
* place within the taxonomy for every member of 
the relevant universe; and (2) when all of the 
subclasses are added together they should 
equal the class from which they were formed. 
In other words, the Strucfure was to be 
. designed so that literally all outcomes of 
postsecondaVy education could be included— 
including process variables that some would 
consider outcomes while others would not, 
such as number of books in the library and 



faculty/student ratio. There should be a place 
in the Structure for any outcome that can be 
conceived of totjiay, as well as a place for 
factor^ that will come to be considered 
outcomes in the future. 

4. No Overlap. I The fourth principle for 
developing thejStructure. was that it include 
mufua/Zy exclusive categorles to the greatest 
extent possible. This tends to conflict 
somewhat with the previous principle because 
attempts at greater comprehensiveness typi- 
cally create more potential for overlap among 
categories. Nevertheless, it was felt that the 
utility of the Structure with respect to 
classification of outcomes woCild be signifi- 
cantly, decreased if the principle of mutual 
exclusiveness were not applied^ 

/ 

5. Reliability. The Outcomes Structure was to be 
subject to the principle of reliability. As used 
^iere, this includes several principles used by 
previous developers of outcome-related classi- 
fications. IBIoom's (1956) "logic** (logical and 
logically ordered subdivision) and "internal 
conslstencv** (terms are to have a consistent 
definition and to be used in a consistent 
manner throughout the classification) are 
subsumed under this term. So is Derr*s (1973) 
"Independent judges** criterion, which requires 
consistency In the classification of specific 
oducational outcomes by different people. 
Also Included is the - principle stated by 
Simpsc" (1961) and other taxonomlsts that 
there be detinlte, separate classes and 
clear-cut definitions for each class, so that 
educational outcomes can be placed into the 
Structure with reasonable ease and consis- 
tency. The consistency of the definitions used • 
is all-important to the success of the 

classification system. 
> 

6. Neutrality. A sixth principle was that the 
Structure and its components be a neutral tool . 
This follows a principle suggested by Bloom 
and his associates (1956). In other words, 
normative issues concerning whether a 
particular outcome is desirable or undesirable 
were not to be treated. Thus, for example, the 
Structure should permit incorporation of both 
positive economic benefits to the community 
from an institutions* presence (such as faculty 
spending) and negative economic benefits 
(such as land removal from the tax base). 
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lijere are several reasons for this position with 
respect to values. The most obvious is the fact 
that^yalqe systems differ and that the 
separation otgood from bad outcomes will be 
accomplished differently by different individ- 
uals and groups. For example' an outcome 
'v sych as liberalization of student a'ttitudes can 
rather easily be seen to be positive or negative 
depending upon one's perspective. To put the . 
matter somewhat more formally, the normative 
c^imenslon was not to be Included In the 
Structure simply because desirability versus 
undesirability is nbt an attribute^ of outc6mes 
in.agd of themselves, but rather resul^from 
the perceptionsT and value systems of those 
^ who encounter such outcomes. A second and , 
somewhat more pragmatic reasori^ is also 
pertinent. The Outcomes Structure/is intended 
, to place some boundaries around the question 
of what constitutes the outcomes of 
postsecondary education. Logically speaking, 
then, it should jnclude both ' positive and 
negative outcomes. Furthermore, the Structure 
is Intended to be a vehicle for consensus 
building with respect to that boundary-setting 
question. Thus, arbitrary imposition of one 
particular set of values at this point in time is 
simply oot appropriate. 

Preventing values from entering into a 
classification system is extremely difficult, if 
riot impossible, but they must be kept out as 
much as possible. The reason for the extreme 
difficulty is that factors other than the terms 
used may introduce the authors* values into the 
' system. In particular, the "dimensions and 
categories used, and the order in which they 
are listed, may be more amenable to and 
suggestive of certain outcomes than others. 
Ornell (1974) has expressed this problem 
especially well: 

The mechanism by which systems of classifi- 
cation Impinge on questions of, value Is, in 
essence, that classifications tend to throw 
emphasis onto certain qualities, and tend to 

^ diminish the apparent significance of others. 

* Behaviors which do not fit conveniently into 
the classification are broken up, and thereby 
apparently diminished In weight. Hegers 
classification of movements of sociafthought 
Into XhBSBs and anti-theses may have looked 
innocuous enough when it was first 
formulated. Surely, It must have^been said. It 
Is a tautology that one can always divide 

^movements of social thought Into these two 
categories? Does this not have the certainty of 
other analytic truths, such as "It is, either 
raining or not raining**? Yet we now see that 



the dichotomy subtly prejudges the Issue of 
whether, at any given^tlme, there Is a definite 
social "thesis;" and it can hardly be disputed 
that its effect ^ to encourage a polarization of 
society into prematurely Identified positions. 
. . . it hardly needs to be said, therefore, that It 
Is important to Ipok at a given classification of 
educational objectives criticaily: to try to 
identify the hidden assumptions embodied In 
it, and to delineate the kinds of value-systems* 
^ Jn education to which it may be usefully 
related. What one cannot do with Intellectual 
credibility Is to assume that the business of 
classifying educational objectives Is a purely 
technical matter, which is Independent of the 
central philosophical questions relating to the 
alms and values of education. 

Turning to Bj6om*s taxonomy of the cognitive 
domain, we find a way of subdividing 
objectives which puts "synthesis** and 
, "evaluation** squarely at the top of the 
hierarchy. It may be argued that these are self- 
evidently "higher" objectives than those of 
analysis and application. But If we see this 
order of objectives as "self-evident,** may we 
rtot be already presupposing the kind of yalue 
system which fits the taxonomy most neatly? 
[pp.3-4] 

7. Hierarchy, A seventh principle used was the 
need for some hierarchy. Various hierarchies 
have been used in previous outcome-related 
classificatioQs (for example, simple to 
complex, concrete to abstract, internalized to 
not internalized, short-term to long-term, 
duality to relativism). The point hece is not that 
one particular hierarchy is better than another 
but rather that the inclusioa of hierarchical 
order has some definite advantages for a 
classification. One of the advantage^ was 
pointed out by Blackwelder (1967) when he 
stated that "literally hundreds of facts** about' a 
particular species are indicated merely by its 
location in the hierarchy of the zoological 
classificatjon. Vickery (1968) noted another: 
"lateraddffions to the array can be interpolated 
helpfully'* (p. 41). 

8. Reality, The Structure was to be based upon 
reality. In other words, the classifications used 
should reflect (to the extent they are shown) 
the most important or essential relationships 
atnong the different educational outcomes. 
Thus, the classification should conform to 
observed relationships .or correlations among - 
educational outcomes, and between outcomes 
and other factors. Furthermore, the classifica- 
tions should conform to the latest, most 
relevant, and most accepted theory about 
different outcomes. Blackwelder (1967) has 
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referred to this as the principle of 
"correspondence"; Simpson (1961) labels it 
"objectivity." 

9. Flexibility. Finally, it was deemed important 
that the Outcomes Structure have functional 
flexibility. Naturally, any classification should 
be designed to have maximum relevance to the 
context In which it is used. As Vickery (1968) 
stated, "characteristics of classes and 
su])classes should be chosen for^their 
relevance to the purpose of the classification, 

^ their ascertainability, and their relative 
permanence" (p. 38). Since the Outcomes 
Structure is intended* for a variety of different 
users, at different levels in an institution and 
outside of the institution, it is important that it 
be flexible and readily adapted by different 
users. It was specifically decided that this 
Structure should be useful for both analysis 
and classification. 



Does the Structure Meet the Evaluative Criteria? 

Only widespread trial use of the NCHEMS 
Outcomes Structure will show how well It meets 
the established criteria. However, revleyvers of the 
Structure provided evaluative information prior to 
the completion of this document such that the 
authors believe the Structure will, for most uses, 
meet the criteria. 

1. Practical Utility. Over 100 people from diverse 
backgrounds provided reactions to the 
Structure. In general, they expressed the 
opinion that it has potential to be of practical 
use to decision makers In postsecondary 
education institutions, and to researchers In 
the area of outcomes, and that It may be useful 
to some decision makers at the state and 
federal levels. A number of faculty members 
expressed the opinion that it could be useful in 
course planning (for which It was not 
designed) as well as In pr^gr^m planning. 
Several students and one student personnel 
administrator expressed the opinion that it 
even could be useful to students in planning 
what they want to get out of college. 

Based on a drMt version of the applications 
document (Lenning, 1977a), the Structure was 
used at the University of Colorado to test the 
adequacy of several outcome lists, developed 
over a period of three years, which their staff 
was confident were comprehensive enough for 



the needs qf the institution. The person who 
coordinated development of the lists was hired 
by NCHEMs to apply the Structure in the 
manner outlined in the draft manual. She had 
no previous knowledge of the Structure or the 
procedures outlined, and was not given any 
orientation to them. Nevertheless, she 
reported later that she Was able to understand 
quickly the.Structure and procedures, and had 
no trouble applying them (though this may be 
accounted for by her extensive previous work 
with outcomes concepts while developing their 
outcome lists). To the staff's surprise, 
application of the Structure revealed that 
several outcome areas they considered to be 
very important had been left out of their lists. 
Based on this discovery, they are revising their 
lists where indicated and modifying their 
freshman student questionnaire accordingly. 
For more information about this case study, 
see Endo arid Lenning (1978). 

Th§ Structure also was tested In a preliminary 
way at eight small liberal arts colleges by a 
project that NCHEMS conducted jointly with 
the Outcomes Learning Task Force of the 
Council forthe Advancement of Small Colleges 
(Lenning, 1977c; Lenning and Lundin, 1977). 
For four of the colleges there were campus 
coordinators for the project who met ahead of 
time as a group with NCHEMS personnel to be 
oriented to the Structure and to be involved in 
planning for the project. 

Each campus was visited by a team composed 
of an NCHEMS staff member and a CASC Task 
Force member:. The visits focused on 
Introducing faculty to the use of the Outcomes 
Structure at departmental and Institution-wide 
levels In relation to student-learning outcomes. 
A workshop of one-half day was used as 
orientation, followed by interviews during 
which the faculty members were asked to use 
the Structure to: 

(1) Identify student groups that should be 
considered separately for planning pur- 
poses within their program or area of most 
concern; ^ — -- 



(2) Rate, for each of those student grpups, the 
relative importance of various type-of- 
outcome categories; 



(3) List specific, concrete oa 
selected categories— Jhat tfiey 
cially important; 



comes— within 
rated espe- 
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(4) Identify specific^ evidence w^ich wouid 
indicate that those outcomes had occurred; 
and ^ 

(5) Identify student experiences and activities 
that wouid do the most to promote those 
specific outcomes. 

The interviews showed that the Structure couid 
stimulate awareness of objectives of a program 
or an institution, as well as increase ability to 
examine outcomes more systematically and 
concretely. Some faculty commented that they 
found the Structure to be an aid in expressing 
the intended results of their work with 
students. A number indicated ways they 
believed the Structure would be useful (see 
Figure 4.2), and, in each case where there was 
time, interviewees were asked to rate the 
usefulness of the Structure. Responses to the 
rating question are shown in Figure 4.3. A 
number of those interviewed also commented 
on cautions and potential problems that might 
occur with use of the structure. These 
comments are listed in Figure 4.4. 

Pairs of CASC Task Force members visited the 
four other colleges and interviewed faculty, 
administrators, and students. At these four 
institutions the interviews started with a 
discussion of student activities, after which an 
attempt was made to' relate those activities to 
the outcome categories of the Structure 
type-of-outcome dimension. Many faculty 
members resisted the attempt to shift the 
focus to outcomes; they felt more comfortable 
;talking about activities. The interviews did note 
that in most cases the Structure performed a 
useful function, however (Lenning and Lundin, 
1977): 

The "CASC approach" was not built around the 
Structure, but utilized it in the attempt to make 
the transition from a focus on activities to 
linking activities to outcome areas. In this 
sense it was centra! to understandings of the 
relations/)ips of activities to the larger 
purposes toward which these activities are 
directed. Without .the Outcomes Structure 
categories, the faculty members could have 
focused on activities without any systematic 
examination of the larger purposes. The 
Outcomes Structure was also noted to serve as 
a taxonomic device for communicating 
outcomes across disciplinary lines, [p. 17] 

One of the participating colleges (Spring Arbor 
College), as a follow-up to the campus visit, 



attempted still another application of the 
Structure. THe campus coordinator. Dr. Ralph 
Sickmiller, used the second-level-of-detail 
categories of the type-of-outcome dimension 
to classify institutional and institutional-unit 
goals each into one of the five first-level 
categories of that dimension. The goals 
consisted of priority Institutional Qoals 
Inventory statements and statements self- 
constructed by institutional faculty and staff. 
For the institution,' and separately for each 
unit, a goal-profile line graph was drawn that 
indicated the number of goal statements found 
for each broad category. Comparisons of 
profiles with one another suggested that this 
procedure may have practical usefulness in 
defining goal differences and commonalities 
among units. 

Other tests of practical utility will be 
necessary to substantiate the usefulness of the 
Structure in other contexts, but these 
preliminary tests proved encouraging with 
regard to trtis (Criterion', r . 

2. Congruency With Other Decision-Making 
Tools. The Structure is, a revision and an 
extension of the NGHEMS Inventory of 
Outcome Variables and Measures, an 
instrument successfully utilized in conjunction 
with the NCHEMS products mentioned in the 
preceding section. Careful consideration was 
given to making the Structure complementary 
to those NCHEMS products. Concerning 
outside decision-making tools, the ETS 
Institutional Goals Inventory, the ETS 
Institutional Functioning Inventory, the ETS 
Student Reactions to College, and the ACT 
Institutional Self-Study Survey were examined 
directly to see if any outcomes measures in 
those instruments could not fit into the 
Structure. The same was true of a large number 
of previous attempts to structure outcomes 
(Lenning, 1977b). In addition, many scales 
from various psychometric instruments re- 
viewed in Buros* Seventh Mental Measurements 
Yearbook (1972) are referred to in Appendix B 
as examples of possible measures of specific 
types of outcomes. At this time the staff sees 
no serious incompatibility with decision- 
making tools developed either inside or 
outside of NCHEMS. However, again the real 
test of this criterion will be in attempts to use 
the Structure. 
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Figure 4.2 



USES FOR THE STRUCTURE THAT VARIOUS INTERVIEWEES REPORTED PERCEIVING 



• Can be an aid in planning our program. 

• Vehicle for curricular change. ^ 

• Helps one be systematic in thinking about outcomes (2;. 

• Encourages emphasis on desired goals rather than activities. 

• Can serve as a stimulus to think in specific outcome terms. 

• Defining goals and objectives for the college, divisions, and course. 

• . Forces us to be more systematic In thinking about what we are trying to accomplish. 

• Useful for proposal development. 

• Setting goals and objectives. 

• Useful in setting goals and prioritizing them on a value basis. 

• Could be useful also for personal planning of college students concrrning what they want from their college 
careers (3). 

• Could help us develop needs assessment focuses. c 

• It has made me stop and think about what we are trying to accomplish, and is making the objectives in the 
' back of my mind conscious. 

• Outcomes Structure may facilitate curricular reorganization by helping to eliminate overlap in content. 

• Should help to increase efficiency. 

• Structure may facilitate generation of a list of contents of a portfolio of what students can do. 

• It is systematic, all-inclusive, adaptable, and helps to prioritize. 

• Could speed the process of planning and development. ' 

• Forces you to think about things you wouldn't otherwise think about in a concrete manner. 

• Helps to specify goals and determine priorities. 

• Can help us understand the why of outcomes. 

• Able to focus more on the inadequacies of programs. 

• Hadn't thought in this way before; made me aware of and knowledgeable about some of the outcomes I am 
striving for. 

• Similar in some respects to the AIDP forms, but better. 

• Useful for dialogue— for raising questions. 

• Can provide stimulation to guide direction. 

• Could be a unique tool for helping to determine what direction we should be going. 

• Gets at the concrete things we value in our thinking and our actions, and balances one against the other. 

• Could be useful to students in evaluating options— at the sophomore or junior level, and especially if 
undecided. 

• Could use it to help define our goals. 

• Spreads things out and serves as a valuable checklist. 

• This project caused me to sort this department out a little bit. 

• This IS probably being done somewhat and systematically in our department already (education 
department), but our range of objectives has been too limited and we have not taken an adequate look at the 
relationships between faculty persons' actions and the desired outcomes. Process 2 for the Structure could 
especially be useful in this case. 

• I have thought in similar ways before and found it useful. I will be having a retreat for my staff in January, 
and am considering use of this device at that retreat. 

a 

Repnnied from Lenning (1977c) 
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3. Comprehensiveness, With the "other** 
category for each dimension, and the fiexibiiity 
to add categories as needed, one shouid be 
able to classify almost any educational 
outcome, both now and in the future. The 
reviewers reported no problem with this; in fact 
they thought it was quite comprehensive. The 
various trial uses of the Structure that were 
conducted also suggest that it is comprehen- 
sive. Process variables that some would 
consider outcorflBs, while others would not, 
can also be categorized--for example, 
introduction of new teaching methodologies 
into the institotion*s programs (codes 3320 and 
5200), number of volumes in the library (code 
4110), and faculty /student ratio (code 5200). It 
is clear, however, that the "research and 
scholarship** section of the Structure (code 
3300) needs to be refined and expanded, 
through additional levels of detail. This may 
also be true of other sections^ for example, 
"organizational format, activity, and operation** 
(code 5200). 

4. No Overlap. The dimensions used have rather 
clear-cut categories with straightforward 
definitions, which should help reduce overlap. 
Most of the reviewers reported no serious 
problems with this criterion, although it is 
impossible to reduce overlap completely. 



especially at the level of the most detailed 
categories. The reviewers did report that they 
noticed some overlap, but felt it was not 
serious. 



Reliability, Clear-cut categories and straight- 
forward definitions should help increase the 
reliability of. classificatioh, but this was not 
borne out in a classification exercise with three 
independent judges. The three judges 
classified a long list of outcomes taken from 
tbe literature, and ended up classifying many 
of the items differently. In some cases, one 
was able to classify to a much more detailed 
level. In other cases, they gave different 
interpretations to what the outcome statement 
meant. (Some of the statements were very 
broad and vague, similar to items often* found 
in institutional mission statements.) Yet, after 
the task was completed and they had a chance 
to discuss it, the judges indicated that had 
they been able to work together and to discuss 
each outcome statement, they were confident 
they would have been able to come to an 
agreement easily. This seemed to the staff to 
be a more important factor. The categories for 
which the judges did have serious problems 
were, modified to alleviate the perceived 
problem. 



Figure 4.3 

EXPRESSED OPINIONS OF RESPONDENTS 
CONCERNING THE USEFULNESS OF THE STRUCTURE ' 





Did Not . 
Discuss It^ 


Definitely 
Useful 


May Be 
Useful 


Not 
Useful 


Said That They Did Not, 
Understand It Enough To 
Judge 


College A 












'Administrators 


5 


11 


3 


3 


1 


Faculty 


10 


13 


9 


0 


2 


Students 


4 


6 


1 


1 


0 



Each number indicates how many people in that group (identified in the left margin) gave a 
particular response (identified by the column heading). 

^ In these interviews, the time allotted for the interview expired before this question could be 
raised. 



From Lennlng and Lundin (1977) 



Figure 4.4 

CAUTIONS AND POTENTIAL PROBLEMS WITH THE NCHEMS OUTCOMES 
STRUCTURE THAT WERE MENTIONED BY VARI0US4NTERVIEWEES^ 



• Need to get students and alumni Involved In outcome planning also. 

• Need to show people how It will help. 

• This kind of self analysis Is very Important, but will It make any difference on the campus? 

• Might work If you have the right setting, like a workshop away from the campus. 

• Don't use it unless you have the time and energy to change. 

• One needs time away from day-to-day pressures In order to Implement this. 

• Resistance to the "bureaucratese" In which the document Is written— I don't want to go to the trouble of using 
It unless you can show me that It Is directly relevant to my department and Its students. 

• Potentially very time consuming— safeguards should be built In to Insure that the time drain is not too great 
{find out the outcomes of the system that can be achieved In a short time). 

• It may promise more than it delivers. 

• L> the system really hierarchical- Is hierarchy the appropriate model? 

• Need funds for the department to develop long term workshdps.to incorporate use of the Structure. 

• If It is presented properly, students could become interested in th^'Structure. 

• I like it. Dynamite! It could be such a long-term thing, however. 

• Difficult to reach consensus on such things— needs a good leader. 

• Takes too much time- what is needed is funds for a week's staff retreat in a retreat setting. 

• More useful at small colleges than elsewhere because we have more of a chance to see and understand ihe 
activities which lead to outcomes (the why of outcomes) and we get to know our students very well. 

• Need to start using this In goal setting at the Institution-wide level. If you start at the department level will go 
in all directions so that when people get together at the institution-wide level will be so far apart will never be 
able to reconcile. 

• It is easierto think of outcomes in specific and concrete terms at the department level, so should start 
applying the Structure to reach consensus at that level before you try to reach some consensus on 
college-wide goals. 

• Need to have total faculty involvement at the departmental level if you are going to make good use of the 
Structure. 

• An excellent Instrument if used and the system Instructions understood and participated In by most of the 
faculty in the small college. The problem may lie In attitudes— a strong disposition and loyalty to the liberal 
studies and classical structure . . . time constraints, and using the Structure as they envision It. Once It is 
learned and implemented, however, the negatives could be reversed and It could speed the processes of 
planning and development. . 

• A problem with the Structure concerns differences in definitions, and it includes Jargonese. 

• As a student majoring in religious education I tend to think In more general terms. Therefore, it is limiting- 
it limits my options. I would prefer a completely open-ended approach. 

• It would be useful to the extent that it serves rather than enslaves. Minimal outcomes should be emphasized. 
Should let more happen be/ond the stated outcomes. 

• I question its real utility, although it is an asset in that It is systematized. 

• I heard from a staff member who attended the workshop yesterday morning that a lot of good ideas were 
presented, but that they were perhaps overly Idealistic. 

• People need to see the connections sooner to their on-going tasks. Needed is a week at a retreat setting 
working with faculty on Institutional goals and objectives, and how to implement them. 

^ Reprinted from Lennir^g 0977c) 
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6. Neutrality, Value-laden words were eliminated 
from the Structure whenever they were noticed. 
In addition, categories within a class were 
listed alphabetically in theStructure, theexcep- 
tloris being "maintenance" and "change" in the 
"type-of-outcome" dimension and both levels 
of the "audjence** dimension, where a logical 
and seemingly neutral progression seemed to 
be called tor. None of the reviewers reported 
problems with this criterion. 

7. Hierarchy. Both dimensions that had 
categories assigned were hierarchical in 
nature. One comment was that this seemed in 
many places like a "make-ishift" hierarchy 
rather than a natural one. Most reviewers did 
not comment on this criterion. An indication of 
how well this criterion is met awaits extensive 
on-campus testing and use in varied contexts. 

8. Reality. The extensive literature reviewed by 
Lenning and associates (1974, 1975) was 
considered In developing the Structure. Most 
reviewers had no problem with the "reality" 
criterion, although some concern was 
expressed that the Structure is quite different 
from the traditional view of outcomes held by 
academicians—for example, one reviewer felt 
that the l<nowledge and understanding section 
in the type-of-outcome dimension should be 
listed before the human characteristic 
outcomes section,- because most faculty 
consider l<nowledge and understanding to be 
more important. In addition, it will be noted in 
Figure 4.4 that one reviewer reported that the 
Structure seemed too idealistic. Widescale 
testing should show ,how "real" the Structure 
appears to be to a greater number of potential 
users. 

9. flexibility. The "other" categories, the ,ease 
with which categories or entire category levels 
can be combined or bypassed, and the 
encouragement that is given to lasers to modify 
and adapt the Structure to meet their needs 
should mean that it is flexible. The reveiwers 
thought it was flexible. But, again, only trial 
use of the Structure will show how well it 
meets the established criterion. 

Conclusion. The reliability criterion was 
definitely not met in the trial use with three 
judges, but such a failure does not seem so 
serious now as it once seemed, especjaMy when 
the.perceptions of the judges are considered. The 
remainder of the criteria seemed generally to be 



met, although to varying degrees. However, as 
fmphasized throughout this section, most of 
them will really not be tested until the Structure is 
tried out on numerous campuses across the 
nation— especially the practical utility, hierarchy, 
reality, and flexibility criteria. Furthermore, some 
of the potential uses of the Structure have not 
been tried out even in a preliminary way. 



Responses to Important Questions 
Posed by Reviewers 

Figure 4.4 shows various perceived problems 
about the NCIHEMS Outcomes Structure raised by 
one or more Interviewees In the joint CASC/ 
NCIHEMS project. Additional questions were 
raised by a few of the other reviewers. A number 
of these questions should be responded to and 
clarifications stated as seen from the perspective 
of the authors. 

1. By breaking outcomes into component types 
and ever more detailed categories, is It not 
possible to focus so much on specifics that 
one loses sight of the overall, combined, whole 
which is more than the sum of the parts? This 
is a real danger, but it does not mean it is 
improper or lacks value to focus on the 
componenets that constitute the whole. 
Rather, it cautions us to constantly, as we 
examine a part, keep in mind the relationship 
of the part to the other parts and the whole. It 
is difficult, if not impossible, to adequately 
describe the whole without focusing also on 
each important component, for example, 
describing or analyzing what it means to be an 
"educated person." 

2. Could not this attempt to objectify outcomes 
and develop standard categories and defini- 
tions lead to a stifling of diversity, innovation, 
and change^-especially if the Structure is used 
by administrators to spell out the outcomes 
desired and the process to meet those 
outcomes in such specific, precise terms that 
there is little room for innovativeness on the 
part of faculty members? Institutions and 
programs can be unique in the clientele served, 
the outcomes that are attempted or attained, 
and the means used to reach those outcomes. 
Preliminary use of the Structure shows that the 
audience dimension provides a comprehensive 
unjyerse of^categories of potential clients from 
which to choose, and that the type-of-outcome 
dimension provides a comprehensive universe 



'to 



of outcomes categories from which to choose; 
Thus, using the iists of categories as a 
checklist couid stimulate consideration of 
specific unique audiences and types of 
outcomes that might not have been considered 
otherwise. 

Although administrators couid use the 
Structure to stifle innovation, they also could 
stimulate faculty to be more specific and 
concrete In delineating the outcomes intended, 
and thus encourage consideration of alterna- 
tive ways of bringing about the outcomes, 
based on the desired end results. The 
Outcomes Structure can clearly be used to help 
stimulate such concreteness in thinking, as 
demonstrated by the joint CASC/NCHEMS 
project. 

3. Would not the process of using the Structure 
take more of my time and energy than I can 
afford? To use the Structure as an aid to 
development of a complete program of goals, 
objectives, and priorities for a complex 
institution and its programs is an extremely 
lengthy pi;ocess. But that process would be 
lengthy whether or not the Structure was used. 
It is possible that the systematic approaches 
developed for using the Structure, the 
definitions and concepts outlined, and use of 
the Structure as a checklist of the universe of 
alternative possible focuses could shorten the 
process appreciably. Futh^armore, certain 
applications of the Structure do not have to be 
time consuming to be useful— for- example, 
stimulating systematic and more-concrete 
thinking by faculty and staff of what they are 
trying to accomplish in their programs and 
courses, of why they are trying to accomplish 
those ends in particular ways, and of how to 
show others that they are accomplishing their 
goals. Similarly, experiences at the University 
of Colorado suggest that one who is properly 
oriented to outcome concepts can use the 
Structure to evaluate the coverage of extensive 
lists of goals relatively quickly and efficiently. 

4. Does not the use of psychological jargon for 
some terms in the Structure diminish its 
usefulness as an aid to communication? 
Jargon of any type should be avoided whenever 
possible, and an attempt was made to avoid its 
use in developing the Structure. But. for 
certain outcomes, particularly student out- 
comes, psychological terminology appeared to 
be the most genaric and descriptive, and the 



most widely accepted. Council for the 
Advancement of Small College visitors to the 
campuses did perceive that the Structure could 
aid in "communicating outcomes across 
disciplinary lines." Nevertheless, ii is to be 
hoped that extensive use of the Structure' will 
suggest better terms in some areas of the 
Structure, 

5. Does not the fact that the code numbers are 
disconcerting to some people reduce the 
Structure's usefulness for them? Possibly, but 
it need not be so. Code numbers for each 
category were included for two purposes: (i) to 
give people a shorthand that would allow them 
to record outcomes and communicate them to 
knowledgeable others verbally or in writing in a 
mort; succinct and efficient manner, and (2) for 
use in outcomes information stprage and 
retrieval systems. For other uses, the codes 
could be ignored. This is what happened in 
follow-up applications of the Structure at 
Spring Arbor College. Interestingly, in the 
CASC/NCHEM^ir}|erviews, when the respon- 
dents referred to particular outcome categories, 
they generally referred to the code numbers 
rather than the category name— for example, 
"Outcome 2240" rather than "expression and 
communication skill outcomes." Thus, some 
people who initially find the codes discon- 
certing may later become accustomed to using 
them and actually find them useful. But users 
of the Structure are encouraged to modify it as 
appropriate for best meeting their context and 
needs. 

6. The orientation of the Structure is foreign to 
the orientation of most educators--they just 
don't think this way. Would it not have been 
better and more acceptable to them if it were 
closer to their orientation in its form and 
content? The authors tried to make the 
Structure as generic, theoretically and philo- 
sophically neutral, and comprehensive in its 
coverage as possible, and this guided the 
development of the Structure. One person who 
raised this question suggested moving the 
knowledge and understanding categories in 
front of the human characteristic outcome 
categories, because "they are more important 
to most educators." But such a change would 
have made the Structure less neutral than 
retaining the categories in alphabetical order. 
The NCHEMS Inventory of Outcome Variables 
and Measures, the forerunner of the Outcomes 
Structure, was closer in its orientation to that 
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of the typical educator who wil! use the 
Structure. However, that very characteristic ied 
to criticism by some that many of the terms 
used were value-laden (it was not philosophi- 
cally and theoretically neutral), that it was too 



selective in its coverage (not comprehensive), 
and that it was too specific in its orientation 
and lacked some internal consistency (not 
generic). 
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APPENDICES 



ApptndixA . 

Previous AMtmpts to ^Structuro** Educational 
Outcomes and Outcome-Related Concepts* 



Classifications Focusing on Outcomes for Individuals 

A. Classifications Focusing on Intellectual Development 

1 . The Harvard List of General Educational Behavioral Goals 

2. The Bloom and Associates Taxonomy of Cognitive Objectives < 

3. Guilford's Structure of Intellect 

4. The f96l Proclamation of the Educational Policies Cpmmission 

5. Taba, Levine, and Elzey's Categories of Thought Processes 

6. Gagn6 Learning Model 

7. The Florida Taxonomy of Cognitive Behavior 

8. Payne's Lists of Cognitive Objectives from Ebel and the AAAS Commission 
on Science Education 

B. Classifications Focusing on Emotional, Cultural, and Social Development 

9. The Cardinal Principles of Education Set Forth in 1918 

10. Bobitt's Ten Goals for Education 

11. The Principal Aims of Education Set Forth in 1938 

12. TheKrathwohl, Bloom,"and h^asiaTaxonomy of Affective Objectives 

13. IHarvey's Proposed Model for Educational Effects on Belief Systems 

14. Crawford and Twelker's Affective Outcomes of Simulation Games 

15. The Klopfer Structure for the Affective Domain in Relation to Science Education 

C. Classifications Focusing on Physical and Psychomotor Development 

16. ^ The Ragsdale Categories of Motor Activities 

17. * Guilford's System of the Psychomotor Abilities , 

18. The Abernathy and Waltz Framewcrl< for Human Movement 

19. The Simpson Taxonomy of Psychomotor Objectives ' 

20. Cratty's Framework for Psychomotor Learning Outputs 

21 . Fleishman's '^Structure" of Psychomotor and Physical Proficiency Abilities 

22. The Kiblef, Barker, and Miles Classes of Psychomotor Objectives 

23. Singer's Model for the Psychomotor Domain 

24. Harrow's Taxonomy for Psychomotor Objectives 



•Excerpted ifom O T LCnntng Prevtoui Attempts to Stfucture iducattOnai Outcume^ an0 Outcome Reiateti Concepts A Comptiatton and fieviev¥ ol the 
Literature BouKJer CotoraOo National Cenier fof Higher Education Management Syiiems a! WiCMl 1977 
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. D. Broader Classifications ^ 

25. ^ General Educational Goals from the Eight-Year Study 
. 26. Payne's Classification of Rath's Early Discussion on Educational Objectives 

27. The Major Types of Educational Objectives Formulated by the Eight-Year 
Study Evaluation Staff 

28. General Education Goals for Members of the Armed Forces 

29. The Clapp Commission Classification of College Outcomes * 

30. Vernon's Educational Attainment Maps . ^ • 

31. 1950 Purposes of Public Education in California* ^ 

32. The Framework Developed by the Mid-Century Committee on Outcomes- 

33. Kavighurst's Developmental Task Framework 

34. A Framework for Objectives in General Education Suggested by the Work of 
Dressel and Mayhew 

35. The Survey of Behavioral Outcomes of General Education In High School 

36. Findley's Ultimate Goals of Education 

37. sGerberich's Ten Types bf learning outcomes 

38. Mayer's Aims of Education 

39. Downey's Conceptual Framework for the Dimensions of the Task of Public 
Education 

40. Schwartz and Tiedeman's Continuum of Behaviors 

41 . Taba's Types of Behavioral Objectives 

42. The Clark-Trow Typology Framework of College Outcome Goals Developed 
by ACT 

43. The Pace and Baird Outcomes— Personality— Environment Framework 

44. Michael and Metfessel's Major Educational Goal Categories 

45. Tyler's 1968 Listing of Purposes of Education 

46. Chickering's Developmental Vectors for the Young Adult * 

47. Astin's Taxonomy of Student Output Measures in Terms of Type of Outcome, 
Type of Data, and Time 

48. The Perry Framework for Student Development 

49. Plowman's Classification System for Educational Objectives 

50. Gronlun(|{^ Classification of Learning Outcomes 

51 . The Research for Better Schools Classification of Educational Objectives 

52. The German "LOT-Projekt" Mode! for Classifying Educational Objectives 

53. Healy and Associates Taxonomy for Performance Objectives 

54. College Student and Alumni Activity and Accomplishment Scales 

55. Alumni Survey College Goal Scales 

56. Impact and Attainment Areas Covered in Pace's Higher Education 
Measurement and Evaluation Kit 

57. Ebel's Command of Substantive Knowledge Framework 

58. Sthaiock's h\>dels for Educational Outcomes 

59. Tri-County Goals Development Project Student Learning Classification System 
Classifications Focusing on Outcomes for Society 

60. Hand, Hoppock, and Zlatchin's Society-Oriented List of Educational Objectives 

61 . Bowen's Categories of Social Benefits of Higher Education 

62. Schalockand Associate's Classification of Outputs of Educational Research 
and Development Efforts 

63. Derr's Taxonomy of Social Purposes of Public Schools 

Classifications Focusing on Outcomes for Both Individuals and Society 

64. Goals for Higher Education of President Truman's Commission on Higher 
Education 
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65. Gross and Grambsch*s Listing of Goals for Universities 

66. Testing Program Advisory Committee Outlines of Outcomes that Need To Be 
Measured . ^ . ' 

67. Brubacher*s^General Educational Aims Derived from Hist9ry, 

68. The AASA Imperatives in Education 

69. Sanford*s Framework of Aims for College Education 

70. The Swedish LIGRU Scheme for Classifying Educational Objectives 

71. , Jeljema's Goals for the Church-Related Liberal Arts College 

72. Goodman's Classifications of Educational Outputs 
- 73. Brown's "Growth" Classification 

74. Plowman's Model for Desired Educational Effects 

75. ^ The ETS Institutional Goals Inventory 

76. Gross's Approach^o Classifying Objectives 

77. Raine's Taxonomy of Community Service Functions for Community Colleges 

78. Derr'S^Pombined Classification of School Purposes 

/ 79^ Carnegie Commission's Purposes oi Higher Education 

80. Lenning and Associates' College Benefits Classification 

81. Lenning's "Benefits Pyramid" 

82. The NCHEMS Inventory of Higher Education Outcome Variables and Measures 

83. Bowen's Target Group Classification of Outcomes 
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Appendix B / 

Definitions and Outcome Measure or indicator Examples 
for the Type-6f-Outcome Subcategories 



Page 

Economic Outcomes 56 

Human Characteristic Outcomes 57 

Knowledge, technology, and Art Form Outcomes 63 

Resource and Service Provision Outcomes 65 

Other Maintenance and Change Outcomes 66 



The code number for each category is given in the left margin (if the focus is on only 
maintenance the fourth digit would become "1,*' or if the focus is on only change the fourth 
digit would become "2"). Category definitions are provided in the next column, and several 
examples of possible indicators or measures of such outcomes are provided in the right-hand 
column for each category. The measures and indicators listed are only illustrative examples. 
The majority of examples given are for individuals, but it should be remembered that 
composites of these can often serve as indicators of communities and other populations. 
Futui^e NCHEMS work will include the development of relatively comprehensive lists of 
indicators for selected categories of the Structure. 
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Category 
Code Number 


1000 ECONOMIC OUTCOMES* 


1100 


Economic Access and Independence Oufco/nos -Outcomes that relate to the entrance into, obtainabillty, flexibility, and levels and 
amounts of rponetary or pecuniary situations, conditions, and characteristics. 




1 

^(Categories) 


(Examples of Outcome Measures or Indicators) 


1110 


Bconomtc Access— The amouni of openness or ease of admittance 
to economic opportunities, advancement. 


Percentage of students obtajning their first full-time job in tho field 
of their choice within a specified time after graduation. 




0 


The number of alternatives for an entry level Job open to minority 

□rOUD Oraduates comnared to minnrltv nrmm nnnnrarttiatA^ 


1120 


Bconomtc Flexibility, Adaptability, and Security—lhe amounts of 
self-sufficiency, liberty, frugality, thrift, self-government, 
confidence, certainty, safeguards, stability, and adjustment that 
are exhibited in economic matters 


Geogru>hic mobility of college graduates compared to thoss not 
attendm^ college. 

Self-report of college graduates about th6 economic security for 
them and their families, and the contribution of college to this. 


1130 


Income and Standard of Living-^ Amount of profits, return on 
investment, necessities and comforts of life, wealth, and other 
Signs of economic "well-being" that are obtained or possessed. 
Included is direct support provided to Individuals and the 
community through local purchases by the educational institution 
and through staff salaries and wages. 


Amount of annuaPand lifetime earnings of those attending college 
compared to those not attending colieoe. 

Average student and/or former student reported scores on scales 
measuring perceptions and evaluations of their current and desired 
socio<^onomlc level. 


1200 


Economic Resources and Cosfs— Outcomes that relate to the amount and type of material, energy, effort, people, organization, and 
other economic assets that are available or that are expended In economic activities and production. 




(Categories) 


(Examples of Outcome Measures or indicators) 


1210 


Economic Costs and Efficiency-^lhe amounts of sacrifice, effort, 
expenditure, and waste present in economic activities and 
production. 

t 


The absenteeism and tardiness on-the-job of college graduates as 
compared to nonstudents 

The number of firms that use the college degree as an Inexpensive 
screening device that allows them to hire qualified employees at 
minimum initial cost to the firm. 


1220 


} 

Economic Resources (including employees)— The assets available 
that can aid economic production, distribution, and gain. 


Percentage of college graduates employed in management 
positions within a specified time after graduation. 


m 




Average number of patents and /or copyrights received per student, 
former student, and/or faculty member 


1300 


Economic Production —OvXcomes that relate fo the creation of goods, services, and economic value. 




(Categories) 


(Examples of Outcome Measures or Indicators) 


1310 


Economic Productivity and Production— The value of goods and 
services that are created or produced by and within specific 
enterprises of "audiences'* or clients of the educational Institution, 
and especially in relation to the resources expended in the 
enterprise. 


Percentage of college graduates who can adequately do their 
personal typing and complete their own Income tax forms as a 
result of having attended college . 

Expert Judges* ratings of the amount of Increased worker 

production and higher worker motivation that results from having 
attended college 


1320 


Economic Services Provided-- Amount and type of direct- 
assistance activities provideb by the educational institution or Its 
subunits in the economic area 


Dollar amount of goods and services bought in the local 
community by the institution. Its staff, and its students. Number 
of hours of consultation In the business area provided to area 
companies and institutions by the university's college of business. 


1400 


Other Economic Outcomes'-- An example would be that a company with a large payroll located in this community rather than another 
similar community because there is a more prestigious college here 


•Many of the 5ul)categorjes for this category were suggested by the economic out^ts classification developed by Goodman (1971). 
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Category 
Code Number 


2000 HUM/^N CHARACTERISTIC OUTCOMES 


2100 


Aspirations— Leve\s, patterns, and directions (In persons, groups, organizations, or communities) of Interests, desires, drives, 
ambitions, goals, and intentions. 






^CXdmpwo Ui wUlCUmO MOaoUlOo Ol in(«ICallOr9/ 


2110 

• 


Desires, Aims, or Goa/s— Places, conditions, things, or other ends 
that individuals and/or groups crave, toward which they have 
ambition, or that they intend to reach because of importance to 
them 


Changes in observed desires from college entrance to graduation. 

Changes In the reported aspirations for graduate school as a class 
proceeds through undergraduate school. 

Self-report of changes In goats and aspirations as a result of 
college. 








12120 


Dislikes, Likes, and /nfez-esfs— The persons or types of persons, 
Objects, content areas, occupations and other things and 
Situations for which there is a preference or antipathy 


The reported likes and dislikes of persons before college as 

(tdmnUfnti to aftAr nraHnatiAn anH ^OmnAriQOn lA^lth •ii/^h /^Hanno 
wMi^/a«wj IV oiiwi yiauuallVll, <snu wmi^.wi lovii wilM oUvii vilcinyo 

Over the same period of time for those the same age not ajit^ndlng 
college. 

Score or change In score on an Interest Inventory, e g.. Strong 
Vocational Interest Blank, Kuder General Interest Survey, Kuder 
Occupational Interest Survey, ACT Interest Inventory * 






Self-report of changes In Interests as a result of college. 


2130 


4 

Motivation or Dnvd Level— The intensity of striving toward a goal 
that IS elicited by a need or other stimulus 


Score or change in score on an instrument that measures "need for 
achievement" or "achievement motivation," e.g., the Strong 
Vocational Interest Blank Academic Achievement Scale, the 
College Student Questionnaire Motivation for Grades Scale, the 
California Psychological Inventory Achievement Scales, Personal 
Value Inventory. 

Self-report of changes in motivation level as a result o? college. 


2140 


Other Asptrationat Outcomes 




2200 


Competence and Skills— Leweis, patterns, and direction of ability, capability, proficiency, and talent of different Kinds. 




(Categories) 


(Examples of Outcome Measures or Indicators) 


2210 


Academic SfciZ/s— The amount of ability or competence In taking 
tests, earning good grades, persisting in college, etc without 
regard to the amount of cognitive learning that nas taken place 


Grades earned when the effect of ability, motivation, and other 
such factors have been controlled. 

Persistence In college when the effects of ability, motivation, and 
other such f^tctors have been cancelled out. 






Score or change In score on a test o* study skills, e.g.. Brown- 
Hottrman Survey of Study Habits and Attitudes, Comprehensive 
rest of Basic Study Skills 


2220 


Citizenship and Family Membership SkiHs—Jt\e ability or 
competence to perform relative to the rights, duties, and privileges 
of a member of a family, community, state or nation; for example 
competence in managing family finances, being an effective 
consumer, and evaluating political issues 


Self-report of abilities pertaining specifically to citizenship and 
home membership that college accentuated 

Evaluation by others of citizenship and home membership skills 
masters exhibited 

Score or chango m score on the VIneland Social Maturity Scales 


*As an example, one could at grddudiion compare interest test scores of college students to a group of their high school classmates not attending 
college who had similar interests in high school As another example, one could look at Interest test change scores for college students, adjusted 
for initial level 
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CatftQOfy 
Code Number 


2000 HUMAN CHARACTERISTIC OUTCOMES (continued) 


2230 


Cr—tMty S/f///s— The amount of ability or competence In 
designing, producing, or otherwise bringing into existence original 
perspectives, explanations, and Implementations. 


Score or change in score on a test that measures originality and 
creative ability, e.g., Minnesota Test of Creative Thinking. Test of 
Creative Ability. Guilford's Alternate Uses Test. Sixteen 
Personality Factors Questtonnalre Creativity Scale. 






Evaluation by judges of creative ability demonstrated in a building 
or forming task. 


2240 


Bxpnsslon and Communication Skills— lt)e amount of ability or 
competence In conveying information, attitudes, or emotions on a 
one-toone basis and /or to large or small groups or populations, 
by whatever media, in order to inform, ctiallenge, uplift, and/or 
persuade, etc.. and In receiving and Interpreting such 
communications— through reading, writing, speaking, listening, 
touching, body movement, silence, and cultural arts like acting, 
painting, sculpturing, singing, playing musical Instruments, etc. 


Score or change In score on tests that measure the ability to 
communicate or express oneself. 

Judges' rating In a debate or speech contest. 

Judges' rating of expression in a music, art. or ballet contest. 


2250 


Intellectual SA///S— The amount of ability or competence In 
formulating and analyzing problems, comprehending and 
understanding, synthesizing information, evaluating Information. 
Implementing a solution to a problem, and In locating, retaining, 
and filtering relevant knowledge. 


Score or change In score on a test that measures ability to analyze 
and soive problems and to make inferences, e.g.. California Test of 
Mental Maturity, Watson-Glazer Critical Thinking Appraisal, 
California Psychological Inventory Intellectual Efficiency Scale. 

Self report of changes in analytical ability as a result of college. 


2260 


Interpersonal, Leadership, and Organizational SA/z/s— The amount 
of ability or competence In effectively living and Interacting with 
others, social organizing, being a congenial friend and companion, 
establishing courses of action for others, and Influencing others to 
follow 


Leadersh Ip awards . 

Self perceptions and evaluation of interpersonal and leadership 
ability. 




Perceptions by Judges of interpersonal and leadership skills. 

Score or change In score on a test that measures leadership and 
interpersonal ability, e.g., California Psychological inventory 
Leadership Scale, Chapin Social Insight Scale. 


2270 


Occupational S*i7/s— The amount of ability or competence In the 
special, unique skills required by particular occupations, and In 
seeking, gaining, and maintaining a particular level and kind of 
employment. 

C 


Spatial relations test scores for someone who Is. or Is going to be, 
an artist. 

Demonstrated ability in writing FORTRAN or COBOL for someone 
who is, or is going to be, a computer programmer. 






Score or change in score on the Bennett Mechanical 
Comprehanslon Test. 


2280 


Physical and Motor SA/7/s— The ability or competence In tasks 
requiring physical c(X)rdlnation. dexterity, manipulation, and other 
muscular or moto: skills. 


Score or change In score on tests that measure motor skills, e.g.. 
Crissey Dexterity Test, Minnesota Rate of Manipulation Test 

Judges' scores on skill everas In athletic competition such as 
gymnastics, diving, and figure skating 




Other Skill Oufcomes — Examples are the ability to teach 
effectively, to handle one's leisure, etc 




2300 


Morale. Satisfaction, and Affective Characteristics— le>fe\s, patterns, and directions of characteristics typified by emotion. 




(Categories) 


(Examples of Outcome Measures or indicators) 

\ 

Score or change In score on an attitude scale, e.g.. Thurstone and 
Chave's Scale for Measuring Attitudes Toward the Church, Col'ege 
SSudent Questionnaire Pari 1, Adorno Ethnocentrism Scale. Shaw 
and Wright Scales for the Measurement of Attitudes 

Self-report of one's attitudes and the effect of college on them. 

Score or change In score on an instrument that assesses values, 
e.g.. Alport 'Vernon .LIndsey Study of Values. Differential Value 
Profile, Work Values Inventory. 

Self 'report of one's values and the effect of college on helping to 
clarify them 


2310 


Attitudes— l\\e disposition or tendency to respond either 
positively or negatively to particular persons or types of persons, 
things, situations, etc It is a predisposition to act in a certain way 
AND 

Vaiuas—A strong preference based on a conception of what is 
desirable, important, and worthy of esteem Values affect an 
Individual's actions and thoughts toward others. 
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Category 

CodeNurrber 


2000 HUMAN CHARACTERISTIC OUTCOMES (continued) 


2320 


B0li0fs, Commitments, and Philosophy of The accepttnce 
and hternalizatlon of particular propositions or declarations; the 
particular things that one Is convinced are true. The held view of 
what ' nian" Is, the purposes and reasons for a person's existence, 
and the system of principles and laws that should govern his/her 
thought, morals, character, and conduct or behavior. Included is 
the promo\!on of and the adherence to the conventions, practices, 
and teachings of religious organizations or sects. 


Score or change In score on instruments that assess beliefs, t g., 
Harvey's Conceptual Systems Test, Inventory of Beliefs. 

Self-report of one's beliefs and commitments and the effect of 
college on them. 

The membership and participation in, and support of, a particuicr 
religious organization or cause prior to as compared with after 
college. 






Self-report of one's philosophy of lUe and the effect of college on 
clarifying and organizing It. 


2330 


Feelings and £mof/ons— The disposition or teno-^ncy to respond or 
not respond subjectively to stimuli and the ability ir> control or not 
control such expressions, I.e., feelings of anguish ?r distress, 
anticipation, anxiety, concern, contentment, empathy, excitc^nt, 
fear, frustration, happiness and Joy, humor, lethargy, love, 
pleasure, satisfaction, sorrow, etc. 


Openness and acceptance of feelings before college compared to 
after college. 

Development of an appreciation of different cultures and a wide 
range of human values as a result of college. 

Greater reported satisfaction with life as a result of college 


2340 


Mores, Customs, and Standards of Concfucf—Soctai and cultural 
practices, rules, and conventions designed to guide personal and 
corporate behavior. They have strong ethical or moral significance 
according to tradition and are enforced by social disapproval of 
violations. 


Self-report of the e*fect of college on assimilation or Intemaiization 
of the customs of community or society. 

Score or change in score on the California Psychological Inventory 
Socialization Scale. 

The adherence to particular mores or social customs prior to 
college as compared to after college. 

The amount of subjectivity end enrK)tion guiding one's standards of 
conduct prior to collegeas compared toafter college. 


2350 


other Affective Outcomes 




2400 


Perceptual Characteristlcs^leiie)s, patterns, and directions of consciousness, awareness, and sensitivity exhibited, and the vlew(s) 
orconcept(s) of self, others, surroundings, events. Ideas, etc. 




(Categories) 


(Examples of Outcome Measures or Indicators) 


2410 


Perceptual Awareness and Sensltlvity-^the amount of conscious- 
ness or awareness of, or sensitivity to. stimuli that are exhibited by 
Individuals or groups. 


Increased sensitivity to needs and emotional cues provided by 
others 

increased alertness to the opportunities confronting one. 


2420 


Perception of Self^lhe view held about .oneself: the character- 
istics that are perceived, I.e.. self concept. 

- - ■ — 


Development of positive seif-reg&rd and self-confidence as a result 
of college. 

Score or change In score on a self-concept scale, e.g.. Adjective 
Check List, Californle Psychological Inventory Self Acceptance 
Scak-. Tennessee Self Concept Scale. 


2430 


Perception of Others— the manner In which other Individuals and 
particular groups of others are viewed or perceived; the 
characteristics that are perceived. 


Reports by observers about how a person's respect for otr»ers has 
changed as a result of college. 

Self-report of how one's view of others has changed as a result of 
college. 


2440 


Perception of T/i/n^s— The view one holds (I.e., the characteristics 
noted) of Ideas or other things being examined with the physical 
senses. 


increased respect for the Ideas of others as result of college. 

Movement as a resutt of college experiences from seeing things as 
all ".black and white" to complex "grays." 


2450 

M. , ! 


Other Perceptual Outcomes 
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Category 
Code Number 


2000 HUMAN CHARACTERISTIC OUTCOMES (continued) 


2500 


PersonaHty and Personal Coping C/)a/acrar/sf/C3— Levels, patterns, and directions of hurnan conditions, factors, and traits related 
specifically to the mind and mental processes (other than skills, knowledge, and understanding). 




(Categories) 


(Examples of Outcome Measures or indicators) 


2610 


Adventurous ness and /n/f/8f/v«— Willingness to take chances and 
risks; how daring an individual is; willingness to take a stand or 
speak out; willingness and capacity to Initiate personal action or to 
t>ecome actively Involved. 


Reports by impartial observers of changes in initiative that aaem to 
have resulted from college attendance. 

Self-roport of the effect of college on one's willingness to take a 
chance, e.g., to take an educated guess on an exam. 

The frequency that one exhibits speaking out On Issues as the 
college career progresses. 


2520 


Autonomy and Independence— JhQ amount of freedom from 
control and influence of others that is exhibited 


Score or change In score on personality scales that measure 
autonomy and Independence, e.g.. Sixteen Personality Factors 
Questionnaire Group*Oependent vs. Self-Sufficient Scale, 
Edwards Personality Inventory Independent In His Opinions Scale, 
College Student Questionnaire independence Scales, Omnibus 
Personality inventory Autonomy Scale. 






Self-report of wllllngKess to volunteer or "stand up for one's rights" 
and the effect of college attendance on such willingness. 


2530 


Dependability and Responsibility— Ihe amount of rellabillty, 
trustworthiness, and accountability for own behavior that Is 
exhibited. 


Reports by observers of changes In depandablllty and 
responsibility that have occurred during college. 

Score or change In score on scales that measure dependability and 
responsibility, e.g., California Psychological Inventory Responsi- 
bility Scale, Edwards Personality inventory Assumes Responsi- 
bility Scale, Sixteen Personality Factors Questionnaire Expedient 
vs. Conscientious Scale. 


2540 


Dogmatism, Authoritarianism, and Opdn-Mlni^Qdness—Jt\e 
annount of open-mlndedness, asserliveness, unassertlveness, 
and/or unquestioning obedience to authority that Is exhibited. 


Reports of expert observers about changes In open-mlndedness 
that have taken place during college. 

Score or change In score on a scale that nteasures dogmatism 
.^nd/or authoritarianism, e g., Rokeach Dogmatism Scale, 
California Psychological Inventory Dominance Scale. Omnibus 
Personality Inventory Religious Orientation Scale. 


2550 


Flexibility and Adaptability— Jtte amount of adjustment to new and 
changing situations and circunr>stanCGS that is exhibited 


Score or change in score on a scale that (neasures flexibility, e.g., 
CaNfomta Psychological Inventory Flexib>!lty Scale, Omnibus 
Personality Inventory Practical Outlook Scale, oixteen Personality 
Factors Questionnaire Practical vs. Imaginative Scale. 






Reports by observers of changes in adaptability and flexibility that 
have occurred during college. 




0 


Setf-report of the effect of college on adaptability and flexibility. 


2560 


Habits— The tendency to perform certain actions or to behave In 
characteristic, autonnatic ways 


Observations by others of changes in habit orientation that have * 
occurred during attendance. 

Self-report of changes in habits that have resulted from college 


2570 


Psychological Functioning— Jtte amount of psychological 
adjustment, contact with reality, self-understanding, and seif- 
actualization (optimum self-reatization) that is exhibited. 


The amount of realization of one's actual strengths and 
weaknesses, and of what Is reality. 

Score pr changes in score on an Instrument that measures 
psychological adjustment, e g . MlnnesotaMultiphasic Personality 
Inventory. Sixteen Personality Factors Questionnaire. Moody 
Problem Check List. 

Reports by expert observers about changes in the psychological 
functioning of Individuals that have occurred during college 
attendance 



ERIC 
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Category 
Code Number 


2000 HUMAN CHARACTERISTIC OUTCOMES (continued) 


25B0 


Tolennc9 and Pers/sfence— The amount of endurance, tenacity, 
forbearancd. patience, and restraint that Is exhibited. 


Observations by others of changes In tolerance And persistence 
during college. 

Score or changes In score on an instrument that measures 
tolerance and persistence, e.g., Edwards Personality Inventory 
Persistence Scale. California Psychological Inventory Tolerance 
Scale. 


2390^ 


Other Personality and Personal Coping Outcomes 




2600 


Physical and Physiological Characteristics ^leivtls, patterns, and directions of human body Ualts and processes (other than sKUl 
functioning). 




(Categories) 


(Examples of Outcome Measures or Indicators) 


2610 


Physical Fitness and Tra/fs— Physical and physiological 
characterl5Mcs such as toughness, endurance, strength, speed, 
flexibility - dexterity, physical energy, muscular control, size, 
vocal charai. eristics, etc. 


Score or change in score on physical fitness tests, e.g.. AAHP£R 
Youth Fitness Tests. Basic Fitness Tests. 

Self-report of ''feeling in better physical shape** as a result of 
college. 


2620 


Physiological Health— Jt\e physical well-being of individuals; how 
well the system of normal bodily operations is funr^k)nlng. 


Medical doctor's health physical examination report at coliege 
entrance compared to at coliege graduation . 

Self-report of the effect of college attendance on how well alumni 
take care of their bodies. 


2630 


Other Physical or Physiological Outconies 




2700 


Status, Recognition, and Certification— lewels, patterns, and direction concerning recognition of accomplishments, power, prestige, 
reputation, etc. 




(Categories) 


(Examples of Outconf)e Measures or indicators) 


2710 


Completion or Achievement Award— A certificate, diploma, or 
some other award for having completed a course or program, for 
some demonstrated proficiency, or for accomplishment of some 
type 

V 


An honorary degree. 
Graduation diploma. 
Alumni achievement award. 
Sales award oraiob promotion. 
Danforth Fellowship Award 
Being named a Rhodes Scholar. 


2720 


Credit Recognition— Formal or informal acknowledgement of work 
completed or of confi;jence, trust, approval, etc. 


Graduate school grades. 

Credit hours given for completing a course. 

By-line credit for a movie, play, book, or article. 

Financial credit ratlr z issued by a bank or credit bureau. 


2730 


Image, Reputation, or Status— Jt\e amount of fame, distinction, 
respect, and standing In the eyes or the professioi>. the 
community, or some other group 


Being on the social register. 

Being listed In Who's Who 

Oral and written acknowledgements from others. 

Being Interviewed by the press, radio, or TV. 

Writing an autobiography that Is published or having a biography 
written about you. 


2740 


Licensing and Certification— Formal written authority that a 
person or firm Is qualified and has met the test to practice some 
sKIIl or speciality occupation. 


Entry into the state bar 

PaU'lng a cosmetology licensing exam. 

Being a certified public accountant. 

An Insurance company that has been licensed to sell In a state. 
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Category 

Code Number 

■» 


?000 HUMAN CHARACTERISTIC OUTCOMES (continued) 


2750 


Obtaining a Job or AdMssion to a Follow-Up Proyrim— Success 
In being selected for a postgraduate employment position or a 
special educational program at a higher level. 


Entrance to a university after graduation from a community 
college. 

Entrance to law, nDedical, or graduate school. 

Being selected by the civil service. 

Being selected for a company executive position. 

Being hired In the special field for which the training applied. 


2760 


Power and/or ^uf/»or/fy— The amount of acknowledged 
authorization orabllity to Influence, command, enforce obedience, 
or set policy as a right of rank, position, delegated Jurisdtotlon, 
skill, strength, wealth, etc. 


Appointment or election to a position of authority. 
Earning promotion to a position of authority. 
Influencing important community or public decisions. 


2770 




Getting acknowledged credit for the Important Job having gotten 
done. 




Job, School, or Ufa Success —Evidence of success In one's 
occupation or career, In graduate or professional school, or In 
some other aspect of one's life that Is covered In any of the above 
categories. 


Self report of success In career. 

Teacher's rating of success in graduate school. 

Employer's rating of overall on-the-job perfomDance. 


2780 


Othar Status, Racognltlon, and Certification Outcomas 




2800 


Social Activities and flofes— Levels, pattems,'-and directions of social functions assumed and carried out. 




(Categories) 


{Examples of Outcome Measures or Indicators) 




Adjustn)ent to Ref/remenf— Altering self and lifestyle to meet the 
needs and adapt to the limitations of the retirement years 


Percentage of college educated retirees reporting productive 
retirement years compared to reports of those who never attended 
college. 






Self-report of the effect of having attended college on the 
retirement years. 


2820 


^/////ar/ons—Flndlng appr ate organizations and institutions to 
join and associate with, ar oeing accepted by them. 


Number of affiliations and changes in affiliations for college 
graduates as compared to those never attending college. 

Self-report of the effect of having attended college on the 
affiliations sought and on theaffiliatior.^ won. 


2830 


Avocational and Social Acttvlttes and ffo/es— Finding, pursuing, 
and achieving rewarding nonwor1< activities, hobbies, and parts to 
play in society, and exhibiting that pattern of behavior that is 
expected of persons having the status that has been earned. 


The social roles and avocations of college graduates as compared 
to those who never attended college. 

Self-report of the effect of having attended college on the 
avocational and social roles sought, and on those practiced. 


2840 


Career and Vocational Activities and fto/es— Exhibiting the 
patterns of behavior expected and/or that are needed for the part in 
the "world of worit" that has been accepted or entered Into 


The career roles of college graduates as compared to these who 
never attended college. 

Reports of employers concerning the advancement and roles of 
college trained employees versus the advancement and 
occupational roles of those who never attended college 


2850 


Cittzenjhip Activities and ffo/os-~Faciiltating and contributing to 
governmental functions and to the overall well being of 
individuals, the community, and larger society 


Percent voting in a municipal or state election 

Financial and other contribvMons given to service organizations. 

Percent running for public office or campaigning for someone who 
is 




Category 
CodaNumbtr 


2000 HUMAN CHARACTERISTIC OUTCOMES (COntinu«j) 


2860 


F^mUy ActMtl0S and /?o/as— Contributing to and facilitating 
family functions, I.e., parent roles, sibling roles, son/daughter 
"^roles. etc. 


The family rotas of coilaoe graduataa as compared to thoaa who 
never attended college. 

Self^report of effect of the college on the roles played in one's 
family. 


2870 


rfionosnips ano nv/ar/onsn/ps^oociiiiy iniorawiiny wiin ■na 
entering into and sustaining Intimate, In-depth, and satisfying 
associations with others. 


r^HArftrtAHatirtt nf frl«nH« And mlstlnruihlnii of coIIaoa ttdtlCAtAd 

people versus those never attending college. 

o6ii'repori Oi loe erreci or coiiege oit rnenasnips •na socw 
relationships. 


2880 


Other Activity and Role Outcomes 




2900 


Other Human Characteristic Outcomes . - 



Category 
Cooe Number 


3000 KNOWLEDGE, TECHNOLOGY. ANO ART FORM OUTCOMES 


3100 


General Knowledge and Understanding*— ^am\\\ajny with, analysis and comprehension of, and appitc«tion of facts and principles 
across broad areas of study— t)r9«(/r/) of knowledge and understanding— as a result of dissemination through educational teaching- 
learning activities. 




(Categories) 


(Examples of Outcome Measures or Indicators) 


3110 


Knowledge and Understanding o* Qenera! Facts *nd 
Term/no/oy y— Knowing about and understanding, and having an 
adequate vocabulary to be able to describe, the reality, existence, 
and circumstances of particular sensory (observed, heard, felt, 
etc.) phenomena, objects, people, products, events, conditions, 
etc., or components thereof. 


Students* scores or changes In score on standardized or classroom 
tests that measure knowledge and understanding of germaA 
terminology and /or facts. For example, the Miller Analogies Test 
focuses entirely on knowledge and understanding of general 
terminology, and tests like the College Level Examination Program 
(CLEP) or the Graduate Record Exam (GRE) general exam include 
coverage of general terminology and facts. 

Students* self-report of knowledge and understanding about 
general terminology and facts. 


3120 


Knowledge and Understanding of General Processes- Knowing 
about and understanding customs, rules and standards for 
judgments, guidelines, processes, methods, procedures, 
techniques, trends, and other ways of applying and niaKIng use of 
terminology and facts. 


Students* scores or changes in score on standardized or classroom 
tests measuring comprehension of general conventions, 
processes, and methodologies. 

Students* grades In a general application survey course. 


313C 


Knowledge and Understanding of General Theory— Knowing about 
and understanding principles and generalizations, theoretical 
formulations, hypotheses, supposition, conjecture, etc. 


Students* scores or changes In score on standardized or classroom 
tests measuring comprehension of general theories In a bread field 
of study. 

Students* grades In a general survey course on theories of 
philosophy. 


3140 


Other General Knowledge and Understanding. 




3200 


Specialized Knowledge and Understanding* —FammarWy wllh. analysis and comprehension of. and application of facts and principles 
In particular specialized fields of study— depth of knowledge and understanding— as a result of disseniL'^fflon through educational 
teaching/learning activities. 




(Categories) 


(Examples of C-Jtcome Measures or Indicators) 


3210 


Knowledge and Understanding of Specialized Facts ^nd 
Terminology— Kno>N\nQ about and understanding, and having an 
adequate vocabulary to be able to describe the reality, existence, 
and circumstances of particular sensory (observed, heard, felt, 
etc.) phenomena, objects, people, products, events, conditions, 
etc.. or components thereof. 


Students* scores or changes in score on standardized or classroom 
tests that measure knowledge and understanding In a narrow, 
specialized area of study. Professional certification and licensing 
exams usually focus on this type of knowledge, as do tests ilka the 
College Level Examination Program (CLEP) subject exams or the 
Graduate Record Exam (GRE) area exams. 

Students' self-report of knowledge and understanding about 
specialized terminology and facts. 




*The subcategories used for this category came from Bloom 0956). 




Category 
Code Number 


3100 KNOWLEDGE. TECHNOLOGY, AND ART FORM OUTCOMES (continued) 


3220 


Knc^wledg^ Md tyndersfind/nyo/S^/t//2ttf Processes— Knowing 
about and understanding customs, rules and standards for 
•judgments, guidelines, processes, methods, procedures, 
techniques, trends, and other ways of apDiyIng and making use of 
tern)inology and facts. 


Students' scores or changes In score on standardized or classroom 
tests measuring comprehension of conventions, processes, 
methodologies, and techniques unique to particular specialized' 
professions or disciplines. 

Students' grades In a specialized professiorwl course or program, \ 


3230 


Knowledge and Understanding of Specialized f/jeory— Knowing 
about and understanding principles and generalizations, 
theoretical formulations, hypotheses, supposition, conjecture, 
etc. 


Students' scores or changes In score on standardized or classroom 
tests measuring comprehension of specialized theoretical 
formulations and models. 

Students' Orades In a enur^ that nnM into Hanth Ahnut /\n» «o 

more theories or models unique to a specialized discipline or 
profession. 


3240 


Other Specialized Knowledge Md Understanding 




3300 


Research and Scholarship-Knowledge and understanding, techniques, and physical products resulting from basic and aoolled 
research and scholarship. ^ 




(Ca'togorles) 


(Examples of Outcome Measures or indicators) 


3310 


Research and Scholarship Knowledge and Understandlng'^lt\e 
discovery, development, preservation, and professional dissemin- 
ation of knowledge and understanding resulting from activities 
corJucted in basic and applied research and scholarship. 


Average number of basic research publications, applied research 
publications, textbooks, or monographs, etc.. per student, former 
student, and/or faculty member over a specific period of time. 

Number of faculty members and/or former students In the 
sciences listed in American Men of Science, 


3320 


Research and Scholarship Prodacfs— A^ipl^ed techniques (for 
example, a new therapy treatment In the field of medicine or a new 
technique In the field of music) and physical products (for 
example, a new or refined serum) developed from basic and/or 
applied research and scholarship. 


Average number of patents and/or copyrights received per student, 
former student, and/or faculty member over a given period of time. 

Average number of awards and citations received per student, 
former student, and/or facutly member (over a given period of 
time) for discovery or development of technological products. 


3400 


Art forms and WorAs-ReproducIng and preserving existing artistic forms and works, and developing new or revised artistic forms and works. 




(Categories) 


(Examples of Outcome Measures or Indlcatnrs) 


3410 


Arc/i/fecft/re— Outcomes Involving the design for construction of 
buildings, landscape, living complexes, etc. 


Number of architectural works completed by students, former 
students, and/or faculty. 

Number of awards and other recognitions received for architectural 
works on the campus commissioned by campus officials. 


3420 


Oance-Outcomes Involving preservation or development of 
forms, works, and performances In the art of dance. 


Number of former students receiving recognition for performances 
In this area. 

Number of students Involved In dance audltk>ns and public 
performances. 


3430 


Debate and Orafory— Outcomes Involving preservation or 
development of forms and performances In the oratory arts. 


Competition record over a period of years of the college's debate 
team. 

The average numt>er of graduates each year who go on to some 
kind of oratorical career. 


3440 


Orama— Outcomes Involving' ^e preservation or development of 
forms, works. and performu in the professional and arrwteur 
theatrical arts. 


The number of students who enter a professional acting career, 
and th) number acting on an amateur basis. 

The number of drama performances put on for the local community 
each year. 


3450 


Literature and Wryf/ny— Outcomes Involving the preservation or 
development of forms and works In the production of prose, verso, 
and other writings. 


The average number of literary works each year published by 
students, former students, and/or faculty members. 

The number of students and faculty each year who have entered a 
formal state or national writing competition. 
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Category 
Code Number 


3000 KNOWLEDGE. TECHNOLOGY. AND ART FORM OUTCOMES (continued) 


^460 


Mus/c— Outcomes Involving the preservation or development of 
forms, works, and performances In the professional and amateur ^ 
theatrical arts. '^-^^ 

V 


The number of musical productions put on each year by the college 
that are open to the public. 

The number of students involved in public music recitals «nd other 
performances. 


3470 


(}reservatlon or development of forms or works In the graphic and 
pictorial arts. 


gallery. 0 

The number of awards won over a certain period of time for 
pldoriai works by students, former students, and faculty 
members. 


3480 


Scu/pft/re— Outcomes Involving the preservatlonor development of 
. forms or works In the carving, chiseling, casting, modeling, or 
other sculpturing areas. 


The number of sculptures that hav»'been commissioned by the 
college and placed throughout the campus. 

The forms of sculpture that have been developed on the campus. 


3490 


Other Fine Arts 




3500 


Other Knowledge, Technology, and Art Form Outcomes 



Category 
Code Number 


4000 RESOURCE AND SERVICE PROVISION OUTCOMES 


4100 


Provision of Facilities and Even fs— The availability, use. and parllclpatlon in campus happenings, buildings, equipment, and other 
resources by students, other Individuals, and particular groups or communities. 




(Categories) 


(Examples of Outcome Measures or Indicators) 


4110 


Provision of Fdc///f/es— Availability and use of campus grounds, 
buildings, rooms, equipment, etc. 


Number of facilities made available to the students during a 
particular period of time. 

Total number of hours each facility was used by people |^ the 
community, and the number of people-hours of use over a specific 
period of time 


4120 


Provision or Sponsorship of Evenfs— Availability arul participation 
In happenings on the campus or off that are provided or sumulated 
by the college or one of Its components. 


The number of people who attended athletic events, cultural 
events, or other events provided and/or sponsored by the college 
In any one year. 

The number of column Inches of newspaper coverage received by 
specific events In local, regional, and national riewspapers. 


4200 


Provision of Direct Services^lt\e availability, use. and receipt by students, other Individuals, and particutiir groups or communities of 
ass'Stance. care, or other service 




^Categories) 


(Examples of Outcome Measures or indicators) 


4210 


Teacft/ng*— Activities and programs designed to instruct and to 
Impart knowledge. si<ilis. attitudes, etc 


Average number o' courses taught and number of contact hours 

per semester In the regular program. 

Extension courses provided in any one calendar year. 


4220 . 


Advisory and Analytic As^/sf/>nce— Activities and programs 
designed for the purpObt^ of (upon request) offering suggestions, 
recommendations, counsel, information, calculations, and 
studies 


Number of advisory and analytic assistance services offered to 
students, staff, and/or to the public. 

Number of person>hours spent by staff In providing this assistance 
over a specific period of time 


4230 


t 

Treatment, Caro. and Referral Servyces— Helping and direct 
assistance services, other than those above, provided by the 
institution, institutional units, and/or Institutional staff ..^ 


The treatment, care, and referral services offered by the Institution 
and ita staff, and health S'^rvlces. day care for children of v^lng 
mothers, counseling, crisis referral, and drug treatment and the 
amount these services are used 

The reported satisfaction of users of these services with the 
treatment and care received. 


'Some people would consider teaching to always be a producer/ facilitator activity that leads to outcomes Others would, however, consider 
teaching to be an output that results from the interaction of faculty, equipment, students, and other educational resources Those who hold the 
firsl viewpoint should just ignore this category 
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ApptndixC 
How One Can Extend the Outcomes Structure 



Suppose that -nnembers of a committee in the 
political science department at Alpha College (a 
fictitious college) were doing curriculum 
planning, and they knew that the members of the 
department were concerned strictly with 
outcomes of teaching for students. Suppose 
further that they decided to limit their "audience** 
focus to: (1) undergraduate students majoring in 
political science and (2) undergraduate students 
taking political science courses who are not 
political science'majors. They, in effect, added an 
additional level' of categories for the audience 
dimension (they have extended this dimension); 
so a three-digit audience code is necessary to 



represent the situation! The "audience** code for 
the first group of students becomes 111 and the 
code for the second group is 1 12. 

The next task for the curriculum committee 
members was to decide what outcomes to focus 
on, and they decided first to limit their focus to 
"knowledge, technology, and art form change** 
and "human characteristic change**— "type-of- 
outcome**<Hdimension codes 3002 and 2002. In 
effect, they decided to limit their focus to the 
following major subcells of the audience/type-of- 
outqome matrix: 



"AUDIENCE** 
DIMENSION 



111. Undergraduate 
Students Majoring 
in Political 
Science 

112. Undergraduate 
Students Taking 
Political Science 
Courses Who Are 
Not Political 
Science Majors 



111.2002 


111.3002 


112.2002 


112.3002 



2002. Human 

Characteristics 
Change 



3002. Knov^fedge, ' 
Technology, 
and Art Form 
Change 



"TYPE-OF-OUTCOME** DIMENSION 



Next the committee members went to the detailed 
subcategories shown in Figure j.4 and described 
in Appendix B for lype-of-outcome codes 2000 
(human characteristics outcomes) and 3000 
(knowledge, technology, and art form outcomes). 
After looking over the code 2000 categories, they 
decided to examine the even more detailed 
subcategories for the following categories: 
aspirations (2100); competence and skills (2200); 
morale, satisfaction, and affective characteristics 
(2300); perceptual characteristics (2400); and 
social roles (2800). Within these categories, they 
then decided to formulate specific outcome 
objectives, separately for both of their student 
groups, in the following subcategories: change in 
interests (2122); change in desires, aims, or goals 
(2112); change in citizenship and fan'.ily 
membership skills (2222); change in creativity 



skills (2232); change in expression and 
communication skills (2242); change in intellectu- 
al skills (2252); change in interpersonal, 
leadership, and organizational skills (2262); 
change in attitudes and values (2312); change in 
mores, customs, and standards of conduct 
(2342); change in perceptual awareness and 
sensitivity (2412); change in career and vocationaj 
roles (2842); and change in citizenship roles 
(2852). For their students majoring in politiC3)^l 
science they decided also to formulate specifi^f 
objectives in the subcategory: change in 
occupational and employability skills (2272). 

Next, the committee went back to the "change in 
intellectual skills** subcategory (code number 
2252) and extended it to even more detailed 
subcategories— adding a fifth digit to the code 
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Science Majors 



111.2002 
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112.3002 
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Characteristics 
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3002. Knov^fedge, ' 
Technology, 
and Art Form 
Change 



"TYPE-OF-OUTCOME** DIMENSION 



Next the committee members went to the detailed 
subcategories shown in Figure j.4 and described 
in Appendix B for lype-of-outcome codes 2000 
(human characteristics outcomes) and 3000 
(knowledge, technology, and art form outcomes). 
After looking over the code 2000 categories, they 
decided to examine the even more detailed 
subcategories for the following categories: 
aspirations (2100); competence and skills (2200); 
morale, satisfaction, and affective characteristics 
(2300); perceptual characteristics (2400); and 
social roles (2800). Within these categories, they 
then decided to formulate specific outcome 
objectives, separately for both of their student 
groups, in the following subcategories: change in 
interests (2122); change in desires, aims, or goals 
(2112); change in citizenship and fan'.ily 
membership skills (2222); change in creativity 



skills (2232); change in expression and 
communication skills (2242); change in intellectu- 
al skills (2252); change in interpersonal, 
leadership, and organizational skills (2262); 
change in attitudes and values (2312); change in 
mores, customs, and standards of conduct 
(2342); change in perceptual awareness and 
sensitivity (2412); change in career and vocationaj 
roles (2842); and change in citizenship roles 
(2852). For their students majoring in politiC3)^l 
science they decided also to formulate specifi^f 
objectives in the subcategory: change in 
occupational and employability skills (2272). 

Next, the committee went back to the "change in 
intellectual skills** subcategory (code number 
2252) and extended it to even more detailed 
subcategories— adding a fifth digit to the code 



ERJC 



67 



8i 



0 



number to distinguish categories at that ievel of 
detail— using Bloom's Taxonomy of Cognitive 
Objectives <see Figure C.1 ) as follows: 

Ability to Translate Political Science 
Literature (code number 22521 ) 

Ability to Interpret the Political Science 
Literature (code number 22522) 

Ability to Extrapolate fro/n the Political 
Science Literature (code number 22523) 

Ability to Apply Political Science 
Literature (code number 22524) 

Ability to Analyze Political Science Data 
(code number 22525) 

Ability to Synthesize Political Science 
Data (code number 225^6> 

Ability to Evaluate Political Effectiveness 
(code number 22527) 

For code SOQO subcategories, the committeq 
members decided to focus on change in general 
knowledge and understanding (3102) for both 
groups of students, and change in specialized 
knowledge and understanding (3202) for only 
those majoring in political science. They also 
* decided to be concerned with all of the types of 
general and specific knowledge and understanding 
listed in Figure 3.4 and Appendix B, but they 
decided that they needed still mgre detailed 
. "knowledge and understanding" subcategories 
than provided there. For those detailed 
"knowledge and understanding" categories, they 
decided to focus on seven subject matter areas: 
(1) business and management, (2) communica- 
tions and debate, (3) economics, (4) group 
process, (5) philosophy and ethics, (6) political 
.science, and (7) public relations. Roc example, 
they wished to eniphasize political science 
subject matter objectives for all three subcate- 
gories of general and specific knowledge and 
understanding, as shown below. (Note that here 
they also added a fifth digit to the. coding scheme, 
where the fifth digit of "6" referred to political 
science subject matter.) 

General and Specific Knowledge and 
' UViderstanding of Facts and Terminology 
In Political Science (code number 31 126 
or 32126) 



General and Specific Knowledge and 
Understanding of Process in Political - 
Science (code number 31226 or 32226) 

General and Specific Knowledge and 
Understanding of Theory in Political 
Science (code number 31326 or 32326) 

It should be emphasized once more that this is a 
ficti^tious case that was meant to be illustrative 
rather than realistic. However, the example does 
suggest a way in which the Structure could be, 
applied and extended. There are a number of other 
specialized, narrowly focused classifications that 
could be used in a similar manner-to extend the ^ 
Structure (see Lenning's 1977b compilation for 
possibilities). For example, a person designing a 
college physical education curriculum or a certain 
course in that area, would undoubtedly include 
"change in physical and nriotor skids (code 
number 2282)" as one of the subcategories for 
which he or she would develop specific 
objectives. And he or^she might find it quite 
useful to break this subcategory into component 
subcategories using one of the ^psychomotor 
taxonomies that have been developed, such as the 
taxonomies developed .Jby Guilford (1958), by 
Simpson (1972), by Cratty (1969), by Fleishman 
(1972), by Singer (1972), or by Harrow (1972). As 
another example, several of the affective (2300) 
categories could be further subdivided into still 
finer categories outlined by Krathwohl, Bloom, 
and Masia (1964): receiving, responding, valuing, 
and organization. Furthermore, each of these has 
still finer subcategories which could form a sixth 
level of specificity. Similarly, a person designing 
a social studies curriculum might wish to 
subcategorize "attitudes and values" according to 
their focus. For example, these subcategories 
could perhaps be divided into the following^ 
subcategories found in the NCHEMS Inventory of 
Higher Education Ojt6ome Variables and 
Measures: attitudes toward and values concerning 
knowledge and skills, political attitudes and 
values, racial/ethnic attitudes and values, 
personal ethics, social conscience, socio- 
economic attitudes and values, and vocational 
attitudes and values. 

Some people needing to extend the Structure on 
one or more dimensions may wish to develop their 
own additional categories of detail rather than use 
categories developed by others, and such an 
approach is certainly proper if the categories are 
valid and meet the person*s^eeds. For example. 
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Figured 

THE BLOOM (1956) TAXONOMY OF COGNITIVE OBJECTIVES 




11. COMPREHENS* N 



I. APPLICATION 



A. Knowledge of Specifics 

1. Knowledge of Terminology 

2. Knowledge of Specific Facts 



B. Knowledge of Wdys and Meains of Dealing with Specifics 

1. Knowledge of Conventions 

2. Knowledge of Trends and Sequences 

3. Knowledge of Classifications and Categories ; 

4. Knowledge of Criteria 

5. Knowledge of Methodology 



C. Knowledge of Universals and Abstractions in the Field 

1 . Knowledge of Principles and Generalizations 

2. Knowledge of Theories and Structures 




A. 


Translation | 




- B. 


Interpretation 




C. 


Extrapolation 



iV. ANALYSIS 



V. SYNTHESIS 



VI. EVALUATION 



a7 Analysis of Eiernents 



B. Analysis of Relationships 




C. Analysis of Organizational Principles 




A. Production of a Unique Communication 



B. Production of a Pl$n or Proi^K5sed Set of Operations 



C. Derivation of a Set of Abstract Relations 
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Jim Manis at the University of Washington— a 
student in a graduate course there that used the 
review edition of this document as one of its basic 
texts— felt strongly that for the ''audience" 
dimension, ''physical disability condition sub- 
populations," should be split into "nonhandi- 
capped," "situationally handicapped " and "perma- 
nently handicapped" in order to be useful in 



planning ways to facilitate educational outcomes 
for handicapped (disabled) people. Then, for the 
two disability categories, he felt another level of 
detail was needed— for example, mobility without 
aids, mobility with aids, mobility only with a 
wheelchair, mobility with a seeing eye dog, and 
so on, for "permanently handicapped." 
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y Appendix D 


Structure "Type^Of-Outcome" Categories Corresponding to Each Category 


of the NCHEIVIS Inventory of Higher Education Outcome Variables and Measures 


OUTCOME CATEGORY OF 
THE NCHEMS INVENTORY > 


CORRESPONDING CATEGORY OR CATEGORIES 
OF THE OUTCOMES STRUCTURE' 
► — II ■ 


1 1.1.00 Kno\vl6do© DovolopfDont 

1 1.1.01 General Knowledge 
1 1 1.02 Specialized Knowledge 


3002 Kjnow lodge. Technology, ftnd Art Form Change Outcomes 
3102 Change in General Knowledge and Understanding 
3202 Change in Specialized Knowledge and Understanding 


119 on ^IllQ nAVAlnnmAnl 

1.1.2.01 Application of Knowledge Skills 

1.1.2.02 Critical Thinking and Reasoning Skills 

1.1.2.03 Creativity Skills 
M.I. 2.04 Communication Skills 

1.1 2.05 Motor Skills 


<4LU4b V/iMinyB in v/( ipoivrii^vno oniii3 

2252 Change In Intellectual Skills 

2252 Change in intellectual Skills 

2232 Change in Creativity Skills 

2242 Change In Expression and Communication Skills 

2282 Change In Physical and Motor Skills 


1 1.3.00 KnowledgeandSkllls Attitudes, Valuesi and Beliefs 
1.1.3.01 Intellectual Disposition 


2312 Change In Attif 'iisand Values 

2322 Chvi.ge In Belis Cpmmitnrients, and Philosophy of Life 

2122 Change in Desires, Alms, or Goals 


1 2.1.00 Social Skills Development 

1 2 1.01 interpersonal Participation 
1 2.1.02 Leadership 
1 2.1.03 Citizenship 


2262 Change In in tar personal. Leadership, tnd Oroanlzatlonal Skills 
22&'? Change ih Interper sona* Leadarship, arvd Organizational Skills 
2262 Change In Interpersonal, Leadership, and Organizational Skills 
2222 Change In Citizenship and Family Membership Skills 


1 2.2.00 Development ^f Social Attitudes, Values, and Beliefs 

1.2.2.01 Political 

1.2.2.02 Racial/Ethnic 

1.2 2.03 Personal Ethics 
1.2.2.04 Social Conscience 

1.2 2.05 Socioeconomic Aspirations 


2312 Change in Attitudes and Values 

2322 Change in Beliefs, Commitments, and Philosophy of Life 

2312 Change in Attitudes and Values 

2312 Change in Attitudes and Values 

2432 Change in Perception of Others ^ 

2312 Change In Attitudes and Values 

9^17 nhjinriA In AlllliiriA« >Iiim ^ * 

2322 Change in Beliefs, Com..,.trT>ents, and Philosophy of Life 
2102 Change In Aspirations 


1.3.1.00 Student Health Development 

1.3 1 01 Physical Health 
1.3 1 02 Mental Health 


2622 Change in Physiological Health 
2572 Change in Psychological Functioning 
2622 Change in Physiological Health 
2572 Change In Psychological Functioning 


1 3 2.00 Student Personal Attitudes. Values, and Beliefs 

1.3.2.01 Religious and Spiritual 

1.3.2.02 Change/Stability 


2312 Change In Attitudes and Values 

2322 Change in Ballets, Commitments, and Philosophy of Life 

?322 Change In Beliefs, Commitments, and Philosophy of Life 

2312 Change In Attitudes and Values 

2442 Change in Perception of Things 

2422 Change In Perception of Sell 


1 4.1 00 Career Preparation 

1.4.1 01 Academic Preparation 
1.4.1.02 Vocational Preparation 

1.4 2.00 Career Attitudes. Values, and Beliefs 

14.2 01 Achievement Orientation 
1 4 2 02 Educational Aspiration 
1 4.2.03 Educational Satisfaction 
1 4.2.04 Vocational Aspirations 


2272 Change In Occupational and Employability Skills 

3102 Channe In General Knowledge and Understanding 

3202 Change In Specialized Knowledge and understanding 

2212 Change in Academic Skills 

2272 Change In Occupational at n umployablllty SkllK 

3102 Change in General Knowledge and Understanding 

^202 Change in Specialized Knowledge and Understanding 

2312 Change in Attitudes and Values 

2322 Change In Beliefs, Commitments, and Philosphy of Life 

21 32 Change In Motivation or Drive Level 

2102 Change In Aspirations 

2332 Change In Feelings and Emotions 

2102 Change In Aspirations 


2 0 0 01 Discovery of New Knowledge 


3312 Change in Research and Scholarship Knowledge and Understanding 


2.0.0 02 Interpretation and Application of New Knowledge 


331 2 Change in Research and Scholarship Knowledge and Understanding 


2 0 0 03 Reorganization of New Knowledge 


3312 Change in Research and Scholarship Knowledge and Understanding 


(Continued) 


(Continued) 


More than one Outcomes Structure category may pertain to a single NCHEMS Outcomes Inventory oategory in order to best conform to that category 
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OUTCOME CATEGORY OF 
THE NCHEMS INVENTORY (Continual) 


CORRESPONDING CATEGORY OR CATEGORIES 
OF THE OUTCOMES STRUCTURE (Continued) 


3.1.0.00 Community Development** 

3.1.0.01 Community Educational Development 
3.1.0.02 Facuity/Staff Educational Development 

3.2.0.00 Community Service** 

3 2 0.01 Extension Services 
3.2.0.02 Personal Services 

3.2.0.03 Extramural Cultural and Recreational Services 

3.2.0.04 Financial Impact on the Community 

3 3.0.00 Longer Term Community Effects** 
3.3.0 01 Sociaiimpact 
^ 3.3.0.02 Economic Impact 


2002 Change in Human Characterlsdcs 
2002 Change In Human Characteristics 
2002 Change in Human Characteristics 

4002 Resource and Service Provision Outcomes 
4202 Provision of Direct Services 
4202 Provision of Direct Services 
4102 Provlslonof Facilities and Events 
4202 Provlslonof Direct Services 
1000 Economic Outcomes 

0002 Change Outcomes 

2002 Human Characteristics Outcomes 

1000 Economic Outcomes 


* 'The "audience" dimension would account for "community" and the "time" dimension of the Outcomes Structure would account for "longer term!' 
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